Skip to main content
Back to Blog

Measure Mobile Performance as a User Journey

A fast API does not make a fast first screen, and a warm-start average from a development phone hides the slow runs users complain about. This guide measures one journey, cold start to a usable home feed, with Macrobenchmark on a physical low-memory Android device and XCTest launch, hitch and memory metrics on iOS. It covers why Android vitals cannot see time to full display, why an XCTest launch loop is warm by Apple's own definition, counting real network requests with an OkHttp network interceptor, shaping the network and waiting out thermal throttling, and a script that gates on p50 and p95 per device class.

Shashank Rawlani

Engineering Leader | Builder | Problem Solver | AI Engineer

Published Oct 18, 202620 min read
View AI-ready Markdown
Dark technical illustration read left to right. A small green phone outline on the left has a ringed green dot on its screen and a grey bar near the bottom with only a short green segment filled. A curve leads from the phone to a vertical green start line, from which twelve horizontal bars run to the right, stacked top to bottom and sorted from shortest to longest. A dashed grey vertical line with an empty grey ring on top crosses all bars just after the start. Further right a solid green vertical line topped by a ringed green dot crosses them too. The top nine bars are green and end before that line in small green dots; the bottom three are orange, run past the green line and end in crossed-out orange rings. From the ringed dot at the top of the green line, a curve and arrowhead lead into a tall green panel holding four stacked cards with pills. Thin grey rules run along the top and bottom and faint pills sit at both edges.
Quick answer: Pick one journey and define its finish line as a moment the app reports, such as the home feed being rendered and tappable, not the first frame. Run it on a physical device from the device class you are worried about, with the network shaped and the device's temperature logged, for enough iterations that a 95th percentile means something. Record startup to full display, frame overrun, peak memory and the number of real network requests for each run. Keep one versioned baseline per device class and fail the build on fixed percentile thresholds. Never gate on a single warm-start average from a development phone.

The dashboard says the home-feed endpoint answers in 180 ms at p95. Support says the app takes "forever" to open on cheap Android phones. Both are true. Between the tap on the icon and a feed the user can scroll sits process creation, class loading, a dependency-injection graph, a database migration, two analytics SDKs, a sync job someone scheduled at launch, image decoding and the first frames of a shimmer animation. The API call is one line in that list.

The usual performance check does not see any of it. Someone launches the app five times on a flagship phone with the app already in memory, averages the numbers, and calls it a baseline. That measures a warm start on hardware your slowest users do not own, and the average hides the slow runs that generate the complaints. This part of the series builds a check around one journey instead: cold start to a usable home feed on a low-memory Android device, with the iOS equivalent alongside. Choosing which devices deserve a row at all was the previous part; this one decides what to measure on them.

Versions checked on 2026-10-14. Jetpack Benchmark (androidx.benchmark) 1.5.0 stable, released 9 September 2026; 1.6.0-alpha01 followed on 7 October. Xcode 27 (27A266a), with Xcode 27.1 RC (27A9275) out since 5 October; iOS 27.0.1 is the current release. OkHttp 5.5.0. Appium UiAutomator2 driver 8.7.0. Apple's MetricKit pages for the iOS 27.2 beta mark the MXMetricManager launch-task calls used below as deprecated in favour of MetricManager.trackLaunchTask, which is new in iOS 27; on an iOS 17 deployment floor the older calls are still the ones you can use everywhere.

"Usable" is a moment the app has to report

Android splits startup into two numbers. Time to initial display (TTID) ends at the first frame. Time to full display (TTFD) ends when the app says it is usable by calling reportFullyDrawn(). The platform measures TTID by itself; if you never report, there is no TTFD at all.

That split matters because the field data you already have only sees the first number. Android vitals flags startup as excessive at 5 seconds or more for a cold start, 2 seconds for warm and 1.5 seconds for hot, and its documentation says it uses TTID. A splash screen that hands over to a skeleton in 800 ms and then waits four seconds for content looks healthy in Play Console. Your users see the four seconds; vitals does not. The lab measurement has to cover the gap the field metric ignores.

The common mistake is reporting too early. If reportFullyDrawn() runs before the system has recorded the first frame, Android reports TTID and TTFD as the same value. A call in onCreate therefore produces a TTFD that is just TTID with a different name:

// Wrong: reports "fully drawn" before the feed exists.
// TTFD collapses to TTID and the slow network load disappears from the metric.
override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContent { HomeScreen(viewModel) }
    reportFullyDrawn()
}

In Compose, attach the condition to the screen that is waiting. ReportDrawnWhen adds a predicate that must be true before the activity's reportFullyDrawn() is called, and the Macrobenchmark metrics guide recommends it, or its siblings ReportDrawn and ReportDrawnAfter, over calling the activity method by hand:

@Composable
fun HomeScreen(viewModel: HomeViewModel) {
    val feed by viewModel.feed.collectAsStateWithLifecycle()

    // Usable means: first page of real items on screen, not the skeleton.
    ReportDrawnWhen { feed.items.isNotEmpty() }

    LazyColumn(Modifier.testTag("home_feed")) {
        items(feed.items, key = { it.id }) { item ->
            FeedCard(item, Modifier.testTag("home_feed_item"))
        }
    }
}

Write the contract down before writing the benchmark: "on a 3 GB Android device at API 31, cold start to first feed item, p95 under 2.4 s; frameOverrunMs at p95 below zero during the first fling; peak anonymous memory under a fixed ceiling; no more than four network requests." The numbers are yours to choose. What makes it a contract is that each clause names a percentile, a device class and a moment the user would recognise.

A cold start on Android is not a cold start in XCTest

The worked example says "cold start", and the two toolchains disagree about what that means.

Macrobenchmark's StartupMode.COLD kills the process between the setup block and the measured block, so every iteration goes through full process creation. COLD runs also call dropShaderCache(), which needs the target app to ship profileinstaller 1.3.0-alpha02 or later, or a rooted device, so compiled shaders are paid for on every iteration. The same scope offers dropKernelPageCache(), documented as "measuring disk-based startup cost" instead of reading disk data the kernel still holds in memory; it is expected to work on API 31 and later or on a rooted device. A cold Android iteration pays for process creation and shaders every time, and can be made to pay for disk reads as well.

Apple's launch-time guide describes a spectrum instead. A launch straight after force-quitting is a warm launch. A cold launch is one after a reboot, or after a memory-hungry app has pushed your app and the frameworks it depends on out of memory. Now look at what an XCTest launch measurement does. XCUIApplication.launch() terminates any running instance before launching again, and a performance test runs its block iterationCount + 1 times, discarding the first iteration to reduce warm-up variance. Each recorded iteration therefore relaunches an app that was running a moment earlier, which is the situation Apple's guide calls warm. The coldest launch in the loop is the one XCTest throws away.

Neither behaviour is wrong; they answer different questions. The consequence is that an Android number and an iOS number for "cold start" are not comparable, and an iOS lab launch time is likely to look better than the 90th-percentile launch time the Xcode Organizer reports from the field. Treat the iOS lab figure as a regression detector for your own code, and the Organizer, which shows typical (50th percentile) and longest (90th percentile) launch times per device, as the measure of what users get.

One Macrobenchmark for the whole journey

Macrobenchmark runs in a separate test module, drives the app as a user would, and captures a Perfetto system trace for every iteration. It refuses to measure a debuggable or non-profileable build, an emulator, or a device with a low battery, and reports an error rather than a misleading number. That refusal is useful: it keeps the measurement off the hardware that would make it meaningless. Build the target from a benchmark build type initialised from release, marked profileable, as the overview describes.

The test below measures the whole journey in one block, so each iteration produces a matched set of signals from the same launch:

@OptIn(ExperimentalMetricApi::class)
@LargeTest
@RunWith(AndroidJUnit4::class)
class HomeJourneyBenchmark {
    @get:Rule
    val rule = MacrobenchmarkRule()

    @Test
    fun coldStartToUsableFeed() = rule.measureRepeated(
        packageName = "com.example.shop",
        metrics = listOf(
            StartupTimingMetric(),                       // timeToInitialDisplayMs, timeToFullDisplayMs
            FrameTimingMetric(),                         // frameOverrunMs (API 31+), frameDurationCpuMs
            MemoryUsageMetric(MemoryUsageMetric.Mode.Max),
            TraceSectionMetric("net:request", TraceSectionMetric.Mode.Count),
        ),
        compilationMode = CompilationMode.DEFAULT,       // installs your Baseline Profile if present
        startupMode = StartupMode.COLD,
        iterations = 25,
        setupBlock = { waitForThermalHeadroom() },
    ) {
        startActivityAndWait()
        check(device.wait(Until.hasObject(By.res("home_feed_item")), 10_000)) {
            "feed never became usable"
        }
        device.findObject(By.res("home_feed")).fling(Direction.DOWN)   // first interaction
    }
}

Several details in it are easy to get wrong. By.res finds Compose nodes only when the app sets testTagsAsResourceId on its root semantics. StartupTimingMetric prints only minimum, median and maximum in the IDE, and the metrics guide tells you to focus on the median; the per-iteration values you need for a tail live in the JSON report. frameOverrunMs exists only on API 31 and later, and positive values are frames that missed their deadline. MemoryUsageMetric is experimental, and in Max mode it reports the largest sample per iteration of ART heap size, anonymous and file-backed RSS, and GPU memory. CompilationMode.DEFAULT installs your Baseline Profile when the app has one; run the same test once with CompilationMode.None() to see what the profile is buying you. On devices older than Android 14, Macrobenchmark has to reinstall the APK to reset compilation state between benchmarks.

Counting the requests that actually hit the network

TraceSectionMetric counts how many times a named trace section appears, so the app needs to emit one per request. Which OkHttp hook you choose decides what is counted. An application interceptor is called once per call, even when a redirect happens or the response comes from cache. A network interceptor is not invoked for responses served from the cache, and it sees each redirect hop separately. For "how many round trips did startup make", you want the network interceptor:

import androidx.tracing.trace
import okhttp3.Interceptor
import okhttp3.Response

class TraceEachNetworkHop : Interceptor {
    override fun intercept(chain: Interceptor.Chain): Response =
        trace("net:request") { chain.proceed(chain.request()) }
}

val client = OkHttpClient.Builder()
    .addNetworkInterceptor(TraceEachNetworkHop())   // not addInterceptor()
    .build()

One trap in the counting mode: the reference says "a measurement of 0 will be reported if the named section does not appear in the trace". Rename the label, or ship a build where the interceptor is not installed, and the benchmark reports zero requests, which reads as a large improvement. Gate on the count being greater than zero as well as below your ceiling.

Bytes are harder to count reliably from inside the app. If you run on Firebase Test Lab, its --performance-metrics option, on by default, records CPU, memory and network usage for the run. That is coarse but independent of your instrumentation, which makes it a good cross-check on the request count.

Holding the device, the network and the heat constant

A percentile only means something if the conditions behind every sample were the same. Three of them drift on their own.

The device class. The JSON report records the model, the SDK level and memTotalBytes in its context block, alongside cpuLocked. Key every baseline on those fields, not on the CI job name. memTotalBytes is a raw byte count; the sample in Google's own CI documentation shows 3,753,299,968, about 3.5 GiB. Bucket on a rounded value and write the rounding rule next to the baseline, so that everyone reads the class name the same way.

The network. A cold start that fetches a feed is partly a network measurement. Test Lab's --network-profile flag applies a named traffic profile such as LTE, and the gcloud reference states that it only works on physical devices, which is where Macrobenchmark has to run anyway:

gcloud firebase test android run \
  --type instrumentation \
  --app app/build/outputs/apk/benchmark/app-benchmark.apk \
  --test macrobenchmark/build/outputs/apk/benchmark/macrobenchmark-benchmark.apk \
  --device model=MODEL_ID,version=31 \
  --network-profile LTE \
  --directories-to-pull /sdcard/Android/media/com.example.shop.benchmark

Pick MODEL_ID from gcloud firebase test android models list by the attribute the row needs, and list the profile names with gcloud firebase test network-profiles list. The pulled directory is the benchmark app's media directory, where Macrobenchmark writes its *-benchmarkData.json and per-iteration .perfetto-trace files by default; the library prints the exact path to Logcat. If you need tighter control than a named profile, point the benchmark build at a stub backend that serves recorded responses with a fixed delay; then a regression in the server cannot show up as a regression in the app.

The temperature. Twenty-five cold starts back to back heat a small phone, and a throttled CPU makes the last runs slower than the first for reasons unrelated to your change. Android's PowerManager.getThermalHeadroom() returns how much of the thermal envelope is in use, where 1.0 means the device is at or forecast to be at severe throttling. Wait for headroom before each iteration and log what you saw:

fun MacrobenchmarkScope.waitForThermalHeadroom(limit: Float = 0.6f, timeoutMs: Long = 120_000) {
    val power = InstrumentationRegistry.getInstrumentation().context
        .getSystemService(PowerManager::class.java)
    val deadline = SystemClock.uptimeMillis() + timeoutMs
    while (true) {
        val headroom = power.getThermalHeadroom(0)   // API 30+
        if (headroom.isNaN() || headroom < limit) {
            Log.i("JourneyBench", "headroom=$headroom status=${power.currentThermalStatus}")
            return
        }
        check(SystemClock.uptimeMillis() < deadline) { "device stayed hot: headroom=$headroom" }
        Thread.sleep(5_000)
    }
}

Two caveats from the platform documentation shape that helper. Calling getThermalHeadroom() much more often than once a second can return NaN, hence the five-second sleep. And some devices return THERMAL_STATUS_NONE whatever their real state, which is why the helper trusts headroom first and only logs the status. Choose the limit from the reading your device gives at rest. If you add the experimental PowerMetric for battery cost, note that it measures system-wide consumption and only works on physical Pixel 6 and newer devices, and the guide asks for a fixed screen brightness and a stable temperature before the loop starts.

The same journey under XCTest

On iOS the finish line is set with an extended launch task. XCTApplicationLaunchMetric(), created with no arguments, measures until the first frame is displayed and every extended launch task started with MetricKit's extendLaunchMeasurement(forTaskID:) has finished. The same task feeds MetricKit's histogrammedExtendedLaunch in the field, so the lab and production numbers end at the same moment:

import MetricKit

enum LaunchTasks {
    static let homeFeed = MXLaunchTaskID("home-feed")
}

@main
struct ShopApp: App {
    init() {
        // Main thread, before the first scene becomes active. At most 16 tasks.
        try? MXMetricManager.extendLaunchMeasurement(forTaskID: LaunchTasks.homeFeed)
    }
    var body: some Scene { WindowGroup { HomeView() } }
}

// In the feed model, on the main actor, once the first page is on screen:
try? MXMetricManager.finishExtendedLaunchMeasurement(forTaskID: LaunchTasks.homeFeed)

Apple's rules for the task are strict: start it on the main thread before or during scene state restoration, or before sceneDidBecomeActive(_:) on the first scene, with no more than 16 tasks, each overlapping another.

import XCTest

final class HomeJourneyPerfTests: XCTestCase {
    func testLaunchToUsableFeed() {
        let options = XCTMeasureOptions()
        options.iterationCount = 25          // runs 26 times, first one discarded
        measure(metrics: [XCTApplicationLaunchMetric()], options: options) {
            XCUIApplication().launch()
        }
    }

    func testFirstFlingOfFeed() throws {
        guard #available(iOS 26.0, *) else { throw XCTSkip("XCTHitchMetric needs iOS 26") }
        let app = XCUIApplication()
        app.launch()
        let feed = app.collectionViews["home_feed"]
        XCTAssertTrue(feed.waitForExistence(timeout: 10))
        measure(metrics: [XCTHitchMetric(application: app), XCTMemoryMetric(application: app)]) {
            feed.swipeUp(velocity: .fast)
        }
    }
}

The XCTHitchMetric(application:) initialiser arrived in iOS 26, so the skip keeps the suite running on an older floor device. XCTMemoryMetric reports the difference in memory between the start and end of the block, which is a growth figure, not a peak like Android's Max mode. Apple's guide for performance tests asks for a scheme that builds with the Release configuration, with "Debug executable" off and code coverage and sanitizers disabled in the test plan. A Debug build with coverage measures the instrumentation as much as the app.

Xcode's own pass or fail is a baseline value plus a maximum standard deviation, and its report graphs the mean. That is a sensible guard against drift and a weak guard against a tail, for the reason the next section shows. Run on a physical device in the device class you care about; Apple notes that profiling on a device gives higher-fidelity measurements than the simulator.

Gate on percentiles, one baseline per device class

Here is what an average hides. Two sets of 25 cold starts on the same 3 GB device: the second build added a sync job at launch that occasionally blocks the feed. The median barely moves, from 1,930 ms to 1,926 ms. The mean rises from 1,951 ms to 2,066 ms, under 6%, so a 10% tolerance on the mean passes. Three runs out of 25 now take between 2.9 and 3.3 seconds, and the p95 goes from 2,160 ms to 3,120 ms.

The script below reads Macrobenchmark's JSON, keys each report by device class, computes nearest-rank percentiles from the runs array, and compares them with a baseline file kept in the repository. It refuses to compare a device class that has no baseline of its own, and refuses to compute a p95 from too few runs:

#!/usr/bin/env python3
"""Compare Macrobenchmark JSON against a per-device-class baseline."""
import json
import math
import sys
from pathlib import Path

MIN_RUNS = 20          # below this, p95 is just "the slowest run or two"
TOLERANCE = 1.10       # fail if a percentile is more than 10% worse


def device_class(ctx):
    build = ctx["build"]
    ram_gb = round(ctx["memTotalBytes"] / 2**30)
    return f'{build["model"]}|{ram_gb}GB|sdk{build["version"]["sdk"]}'


def pct(runs, p):
    ordered = sorted(runs)
    rank = math.ceil(p / 100 * len(ordered))   # nearest-rank
    return ordered[max(rank, 1) - 1]


def main(baseline_path, *result_paths):
    baseline = json.loads(Path(baseline_path).read_text())
    failures = []
    for path in result_paths:
        report = json.loads(Path(path).read_text())
        cls = device_class(report["context"])
        if cls not in baseline:
            failures.append(f"{cls}: no baseline for this device class")
            continue
        for bench in report["benchmarks"]:
            for metric, limits in baseline[cls].get(bench["name"], {}).items():
                runs = bench["metrics"][metric]["runs"]
                if len(runs) < MIN_RUNS:
                    failures.append(f"{cls} {metric}: {len(runs)} runs < {MIN_RUNS}")
                    continue
                for p in (50, 95):
                    now, base = pct(runs, p), limits[f"p{p}"]
                    verdict = "FAIL" if now > base * TOLERANCE else "ok"
                    print(f"{verdict:4} {cls} {bench['name']} {metric} "
                          f"p{p} {now:.0f} vs {base:.0f}")
                    if verdict == "FAIL":
                        failures.append(f"{cls} {metric} p{p}")
    for f in failures:
        print("GATE:", f)
    return 1 if failures else 0


if __name__ == "__main__":
    sys.exit(main(*sys.argv[1:]))

Run under Python 3.14.5 against the baseline {"Pixel 3|3GB|sdk31": {"coldStartToUsableFeed": {"timeToFullDisplayMs": {"p50": 1930, "p95": 2160}}}}, with the regressed build's report and a report from a Pixel 8 that someone added to the same job, it prints:

ok   Pixel 3|3GB|sdk31 coldStartToUsableFeed timeToFullDisplayMs p50 1926 vs 1930
FAIL Pixel 3|3GB|sdk31 coldStartToUsableFeed timeToFullDisplayMs p95 3120 vs 2160
GATE: Pixel 3|3GB|sdk31 timeToFullDisplayMs p95
GATE: Pixel 8|7GB|sdk36: no baseline for this device class

The exit status is 1. The run data in that example is constructed in the documented report format to show the arithmetic; the script and its output are real. The second failure matters as much as the first. Pool the fast phone's runs with the slow phone's and the combined p95 drops, hiding the regression on exactly the device class the contract was written for.

Two costs come with this design. Nearest-rank p95 over 25 runs is the second-slowest value, so two unlucky runs move it; 20 runs is a floor, not a target, and noisier journeys need more. And cold iterations with thermal waits are slow, so this belongs in a nightly job on a dedicated device, with pull requests getting a shorter smoke run that checks the benchmark still executes. Keep the baseline file and the Perfetto trace of the iteration closest to p95 together as build artifacts: the number says that something regressed, the trace says where.

Pattern-matching across many traces is a good use for a model: clustering which trace sections grew between two builds, or noticing that every slow run overlaps the same background job. It should not move the threshold or decide that a failing run was noise. Fixed devices, repeated runs and the deterministic comparison above make that call. If a separate backend timing question comes up while reading traces, the API Response Time Analyzer helps you keep server latency apart from on-device delay.

Three ways a performance number misleads

A warm-start average from a development phone

Signature: the number is stable and good, and field reports disagree. Check the startup mode and the device. A WARM run "restarts activities without restarting the process", so it never pays for process creation or for loading the app's classes again. Rerun as StartupMode.COLD on the low-memory class with 25 iterations; if the p95 is far above the median, the average was hiding your users.

Device classes mixed in one baseline

Signature: the baseline improves when a device is added to the pool, or jumps when a farm swaps hardware. Check whether the reports in one comparison share a model, SDK level and RAM bucket. If they do not, split them; one baseline per class is the only way a regression on the slow class stays visible.

Animation and background work left running

Signature: TTFD is fine but the first fling stutters, or results vary with time of day. Open the trace of a slow iteration and look for work that is not the journey: a shimmer placeholder still animating behind the loaded feed, a sync job scheduled at startup, an SDK warming its cache. frameOverrunMs at p95 and p99 shows the stall; wrapping the suspect work in its own trace section and adding a second TraceSectionMetric in Sum mode puts a number on it. The fix is to stop or defer that work, not to raise the threshold.

Record one cold-start baseline this week

Take the journey users complain about most, usually launch to the first useful screen. Add ReportDrawnWhen on Android or an extended launch task on iOS at the point that screen becomes usable, and check in Logcat that the Fully drawn line now differs from Displayed. Then run the cold-start benchmark for 25 iterations on one physical device from your lowest RAM bucket, with the thermal wait in place.

Capture three things and commit them: the benchmarkData.json with its device context, a baseline file with p50 and p95 for that device class, and the Perfetto trace of the iteration nearest p95. That is the first versioned journey trace. The next pull request that touches startup gets compared against it, on the same class of device, by a script rather than by eye.

Questions about measuring mobile journeys

Can an Appium suite measure this instead?

Only coarsely. The UiAutomator2 driver's mobile: getPerformanceData returns battery, CPU, memory or network samples parsed from dumpsys, which is useful for spotting a memory climb across a long session. Timing an Appium command measures the command, including the round trip through the driver, not when a frame reached the screen. Keep Appium for functional journeys and use Macrobenchmark or XCTest metrics, which read the system trace or the launch signal directly, for the numbers you gate on.

Why not run the benchmark on an emulator in CI?

Macrobenchmark returns an error on an emulator unless you suppress it, and Google's CI guidance says emulator numbers are tied to the host OS and hardware rather than to a user's device. An emulator is fine for the pull-request smoke run that proves the benchmark still compiles and finds its elements, with that error suppressed through the androidx.benchmark.suppressErrors instrumentation argument. The numbers that feed the baseline come from physical devices you own or rent.

How many iterations are enough?

Enough that the percentile you gate on is not one run. With nearest rank, the p95 of 20 runs is the second-slowest run and the p95 of 40 runs is the third-slowest; the more runs sit above the percentile, the less one bad iteration can move it. If the run takes too long, cut the journey's scope before cutting iterations, and remember XCTest adds one discarded run to whatever iterationCount you set.

Primary references