MacrobenchmarkRule


class MacrobenchmarkRule : TestRule


JUnit rule for benchmarking large app operations like startup, scrolling, or animations.

    @get:Rule
val benchmarkRule = MacrobenchmarkRule()

@Test
fun startup() = benchmarkRule.measureRepeated(
packageName = "com.example.my.application.id"
metrics = listOf(StartupTimingMetric()),
iterations = 5,
startupMode = StartupMode.COLD,
setupBlock = {
pressHome()
}
) { // this = MacrobenchmarkScope
val intent = Intent()
intent.setPackage("mypackage.myapp")
intent.setAction("mypackage.myapp.myaction")
startActivityAndWait(intent)
}

See the Macrobenchmark Guide for more information on macrobenchmarks.

Summary

Public constructors

Public functions

open Statement
apply(base: Statement, description: Description)
Unit
measureRepeated(
    packageName: String,
    metrics: List<Metric>,
    compilationMode: CompilationMode,
    startupMode: StartupMode?,
    iterations: @IntRange(from = 1) Int,
    setupBlock: MacrobenchmarkScope.() -> Unit,
    measureBlock: MacrobenchmarkScope.() -> Unit
)

Measure behavior of the specified packageName given a set of metrics.

Unit
@ExperimentalBenchmarkConfigApi
measureRepeated(
    packageName: String,
    metrics: List<Metric>,
    iterations: @IntRange(from = 1) Int,
    experimentalConfig: ExperimentalConfig,
    compilationMode: CompilationMode,
    startupMode: StartupMode?,
    setupBlock: MacrobenchmarkScope.() -> Unit,
    measureBlock: MacrobenchmarkScope.() -> Unit
)

Measure behavior of the specified packageName given a set of metrics, with a custom PerfettoConfig.

Unit
@ExperimentalBenchmarkConfigApi
measureRepeated(
    packageName: String,
    metrics: List<Metric>,
    iterations: @IntRange(from = 1) Int,
    perfettoConfig: PerfettoConfig,
    compilationMode: CompilationMode,
    startupMode: StartupMode?,
    setupBlock: MacrobenchmarkScope.() -> Unit,
    measureBlock: MacrobenchmarkScope.() -> Unit
)

This function is deprecated. Deprecated in favour of a variant that accepts experimental config

Public constructors

MacrobenchmarkRule

Added in 1.1.0
MacrobenchmarkRule()

Public functions

apply

Added in 1.1.0
open fun apply(base: Statement, description: Description): Statement

measureRepeated

fun measureRepeated(
    packageName: String,
    metrics: List<Metric>,
    compilationMode: CompilationMode = CompilationMode.DEFAULT,
    startupMode: StartupMode? = null,
    iterations: @IntRange(from = 1) Int,
    setupBlock: MacrobenchmarkScope.() -> Unit = {},
    measureBlock: MacrobenchmarkScope.() -> Unit
): Unit

Measure behavior of the specified packageName given a set of metrics.

This performs a macrobenchmark with the below control flow:

    resetAppCompilation()
compile(compilationMode)
repeat(iterations) {
setupBlock()
captureTraceAndMetrics {
measureBlock()
}
}
Parameters
packageName: String

ApplicationId / Application manifest package name of the app for which profiles are generated.

metrics: List<Metric>

List of metrics to measure.

compilationMode: CompilationMode = CompilationMode.DEFAULT

Mode of compilation used before capturing measurement, such as CompilationMode.Partial, defaults to CompilationMode.DEFAULT.

startupMode: StartupMode? = null

Optional mode to force app launches performed with MacrobenchmarkScope.startActivityAndWait (and similar variants) to be of the assigned type. For example, COLD launches kill the process before the measureBlock, to ensure startups will go through full process creation. Generally, leave as null for non-startup benchmarks.

iterations: @IntRange(from = 1) Int

Number of times the measureBlock will be run during measurement. Note that total iteration count may not match, due to warmup iterations needed for the compilationMode.

setupBlock: MacrobenchmarkScope.() -> Unit = {}

The block performing app actions each iteration, prior to the measureBlock. For example, navigating to a UI where scrolling will be measured.

measureBlock: MacrobenchmarkScope.() -> Unit

The block performing app actions to benchmark each iteration.

measureRepeated

@ExperimentalBenchmarkConfigApi
fun measureRepeated(
    packageName: String,
    metrics: List<Metric>,
    iterations: @IntRange(from = 1) Int,
    experimentalConfig: ExperimentalConfig,
    compilationMode: CompilationMode = CompilationMode.DEFAULT,
    startupMode: StartupMode? = null,
    setupBlock: MacrobenchmarkScope.() -> Unit = {},
    measureBlock: MacrobenchmarkScope.() -> Unit
): Unit

Measure behavior of the specified packageName given a set of metrics, with a custom PerfettoConfig.

This performs a macrobenchmark with the below control flow:

    resetAppCompilation()
compile(compilationMode)
repeat(iterations) {
setupBlock()
captureTraceAndMetrics {
measureBlock()
}
}

Note that a custom PerfettoConfigs may result in built-in Metrics not working.

You can see the PerfettoConfig used by a trace (as a text proto) by opening the trace in ui.perfetto.dev, and selecting Info and Stats view on the left panel. You can also generate a custom text proto config by selecting Record new trace on the same panel, selecting recording options, and then clicking Recording command to access the generated text proto.

Parameters
packageName: String

ApplicationId / Application manifest package name of the app for which profiles are generated.

metrics: List<Metric>

List of metrics to measure.

iterations: @IntRange(from = 1) Int

Number of times the measureBlock will be run during measurement. Note that total iteration count may not match, due to warmup iterations needed for the compilationMode.

experimentalConfig: ExperimentalConfig

Configuration for experimental features.

compilationMode: CompilationMode = CompilationMode.DEFAULT

Mode of compilation used before capturing measurement, such as CompilationMode.Partial, defaults to CompilationMode.DEFAULT.

startupMode: StartupMode? = null

Optional mode to force app launches performed with MacrobenchmarkScope.startActivityAndWait (and similar variants) to be of the assigned type. For example, COLD launches kill the process before the measureBlock, to ensure startups will go through full process creation. Generally, leave as null for non-startup benchmarks.

setupBlock: MacrobenchmarkScope.() -> Unit = {}

The block performing app actions each iteration, prior to the measureBlock. For example, navigating to a UI where scrolling will be measured.

measureBlock: MacrobenchmarkScope.() -> Unit

The block performing app actions to benchmark each iteration.

measureRepeated

@ExperimentalBenchmarkConfigApi
fun measureRepeated(
    packageName: String,
    metrics: List<Metric>,
    iterations: @IntRange(from = 1) Int,
    perfettoConfig: PerfettoConfig,
    compilationMode: CompilationMode = CompilationMode.DEFAULT,
    startupMode: StartupMode? = null,
    setupBlock: MacrobenchmarkScope.() -> Unit = {},
    measureBlock: MacrobenchmarkScope.() -> Unit
): Unit
Parameters
perfettoConfig: PerfettoConfig

Configuration for Perfetto trace capture during each iteration. Note that insufficient or invalid configs may result in built-in Metrics not working.