# Build Benchmarks

This directory contains the build benchmarking system. It uses [gradle-profiler](https://github.com/gradle/gradle-profiler) to measure and track build performance over time.

## Overview

The build benchmark system measures various build scenarios to track performance metrics such as:
- **Total execution time**: Complete build duration
- **Gradle configuration time**: Time spent in the configuration phase (task start)

Results are processed to calculate statistical metrics (mean, median, standard deviation, min, max) and can be reported as Pixels for tracking and visualization.

## Components

### `run-benchmark.py`

The main Python script that orchestrates the benchmarking process. It:

1. **Runs gradle-profiler** with the scenarios defined in [build-benchmark.scenarios](./build-benchmark.scenarios)
2. **Processes results** from the generated CSV files
3. **Calculates statistics** (mean, median, standard deviation, min, max) for each scenario
4. **Displays a formatted summary** in the terminal
5. **Optionally reports results** to an external API endpoint

#### Usage

The script can be run from anywhere in the repository:

```bash
python3 build-benchmarks/run-benchmark.py [OPTIONS]
```

#### Command-Line Options

- `--gradle-user-home PATH`: Specify the Gradle user home directory (default: `~/.gradle`)
- `--github-action-run-id ID`: GitHub Action run ID for linking results with CI workflows
- `--git-commit-sha SHA`: Git commit SHA for linking results to specific code versions
- `--report-pixel`: Enable sending results to the reporting API (requires `--github-action-run-id` and `--git-commit-sha`)
- `--skip-profiler`: Skip running gradle-profiler (only process existing results)
- `--skip-processing`: Skip processing results (only run gradle-profiler)
- `--csv-file PATH`: Path to CSV file for processing (only used with `--skip-profiler`). Defaults to `build-benchmarks/results/benchmark.csv`

#### Examples

Make sure that you have [gradle-profiler](https://github.com/gradle/gradle-profiler) installed.

If your Gradle installation is not in `~/.gradle`, provided that as a parameter as well, see `--gradle-user-home PATH` above.


**Run full benchmark locally:**
```bash
python3 build-benchmarks/run-benchmark.py
```

**Run benchmark and report to API (this should only be run from a CI in controlled conditions):**
```bash
python3 build-benchmarks/run-benchmark.py \
  --github-action-run-id "12345" \
  --git-commit-sha "abc123" \
  --report-pixel
```

**Process existing results without re-running:**
```bash
python3 build-benchmarks/run-benchmark.py --skip-profiler
```

**Run profiler only (skip processing):**
```bash
python3 build-benchmarks/run-benchmark.py --skip-processing
```

#### Adding New Scenarios

To add a new scenario, edit [build-benchmark.scenarios](./build-benchmark.scenarios) and add a new scenario block. Refer to the [gradle-profiler documentation](https://github.com/gradle/gradle-profiler) for available options.

Example:
```
build-new-scenario {
    title = "New Scenario"
    tasks = ["app:assembleDebug"]
    warm-ups = 3
    iterations = 5
}
```

## CI Integration

### GitHub Actions Workflow

The benchmarks are automatically run via the GitHub Actions workflow defined in [`.github/workflows/build-benchmark-nightly.yml`](../.github/workflows/build-benchmark-nightly.yml).

The workflow runs **nightly** on the `develop` branch.

It uses the `android-large-runner` which should not be changed. Doing so, or if the underlying resources available to the runner change, will reset our build metric baselines.

#### Manual Triggering

The workflow can be manually triggered from the GitHub Actions UI:

1. Navigate to the **Actions** tab in the GitHub repository
2. Select **Build Benchmark - Nightly** from the workflow list
3. Click **Run workflow**
4. Select the branch you want to benchmark (defaults to `develop` if not specified)
5. Click **Run workflow** to start the benchmark

**Note**: When manually triggered, the workflow will not publish results to the Pixels repository. You can inspect terminal output for see results and compare against baseline or other builds.

## Results

### Output Location

Benchmark results are stored in `build-benchmarks/results/`:
- `benchmark.csv`: Main results file in long format
- Additional files may be generated by gradle-profiler
