Performance Sizing, Reference Architectures, and Large-Instance Tuning: Guided Hands-On Workflow

1. Scenario and preflight: a bounded lab, not a synthetic “production benchmark”

The goal is not to publish a universal benchmark number. It is to practice the measurement discipline you would use in production while keeping every mutation inside resources you own. The lab uses the exact Community Build release currently published at this chapter recheck, a supported PostgreSQL major version, a synthetic Python repository and a stable project key. No proprietary code, external CI provider, paid feature or real credential is required.

2. Start the isolated server and prove its identity before analysis

Create a new empty directory such as sq31-capacity-lab. Save the Compose file below as compose.yaml. The password is intentionally fake and local-only; do not reuse it elsewhere. The SonarQube image tag is pinned to the current Community Build. PostgreSQL is major-version pinned, so the preflight explicitly records its exact patch and image digest.

name: sq31-capacity-lab
services:
  db:
    image: postgres:17
    container_name: sq31-postgres
    environment:
      POSTGRES_USER: sonar
      POSTGRES_PASSWORD: local-only-sq31-db
      POSTGRES_DB: sonar
    volumes:
      - sq31_db:/var/lib/postgresql/data

  sonarqube:
    image: sonarqube:26.9.0.129388-community
    container_name: sq31-sonarqube
    depends_on:
      - db
    environment:
      SONAR_JDBC_URL: jdbc:postgresql://db:5432/sonar
      SONAR_JDBC_USERNAME: sonar
      SONAR_JDBC_PASSWORD: local-only-sq31-db
    ports:
      - "9000:9000"
    volumes:
      - sq31_data:/opt/sonarqube/data
      - sq31_logs:/opt/sonarqube/logs
      - sq31_extensions:/opt/sonarqube/extensions
volumes:
  sq31_db:
  sq31_data:
  sq31_logs:
  sq31_extensions:

docker compose up -d

# Wait until the web process reports UP, then record identities.
curl -fsS http://localhost:9000/api/system/status
curl -fsS http://localhost:9000/api/server/version
docker image inspect sonarqube:26.9.0.129388-community --format '{{json .RepoDigests}}'
docker image inspect postgres:17 --format '{{json .RepoDigests}}'
docker exec sq31-postgres psql -U sonar -d sonar -c 'select version();'
docker stats --no-stream sq31-sonarqube sq31-postgres

Log in only to this disposable instance, change the default administrator password, create the project(s) used by the lab, and create a local analysis identity with only the permissions required to execute analysis on those projects. Export its token through SONAR_TOKEN; do not place it in sonar-project.properties, screenshots, Git history or the evidence packet.

# PowerShell — paste the local token only into this process environment.
$env:SONAR_HOST_URL = "http://localhost:9000"
$env:SONAR_TOKEN = "<LOCAL_LAB_TOKEN>"
sonar-scanner -v

# Bash equivalent
export SONAR_HOST_URL='http://localhost:9000'
read -s -p 'Local Sonar token: ' SONAR_TOKEN; export SONAR_TOKEN; echo

3. Generate a deterministic repository and freeze the baseline revision

The fixture generator produces many small Python files with predictable structure. We use file count as a controlled workload factor because it affects scanner indexing and the amount of analyzed source without introducing external dependencies. This is not a claim that “N Python files equals N LOC of Java.” It is deliberately one synthetic shape whose limitations remain in the evidence packet.

from pathlib import Path
import sys
count = int(sys.argv[1])
root = Path('src')
root.mkdir(exist_ok=True)
for old in root.glob('module_*.py'):
    old.unlink()
for i in range(count):
    (root / f'module_{i:04d}.py').write_text(
        f"def normalize_{i}(value):\n"
        f"    if value is None:\n        return 0\n"
        f"    return (value * {i+1}) % 997\n"
        f"\ndef bucket_{i}(items):\n"
        f"    return [normalize_{i}(x) for x in items]\n",
        encoding='utf-8'
    )
print(f'generated {count} Python files')

mkdir sq31-fixture
cd sq31-fixture
git init
git config user.email "sq31@example.invalid"
git config user.name "SQ31 Local Lab"
python ../make_fixture.py 120
printf "sonar.projectKey=sq31-capacity\nsonar.sources=src\nsonar.sourceEncoding=UTF-8\n" > sonar-project.properties
git add .
git commit -m "baseline 120-file fixture"
git tag sq31-baseline
git rev-parse HEAD

Write the resulting commit SHA into evidence/assumptions.md. The first run is a warm-up run because scanner/analyzer caches, JIT compilation, database caches and OS page cache may not resemble later runs. Do not silently discard it; label it warm-up and keep it so reviewers can see that the test design accounted for cache state.

4. Run the baseline and preserve the asynchronous evidence chain

The PowerShell runner below creates a timestamped evidence directory, records the revision and scanner version, measures client-side elapsed time, preserves report-task.txt, polls the exact CE task until terminal state and records a host/container snapshot. It never treats scanner exit code alone as “analysis complete.”

$ErrorActionPreference = "Stop"
$ProjectKey = "sq31-capacity"
$Run = Get-Date -Format "yyyyMMdd-HHmmss"
$Evidence = Join-Path $PWD "evidence/$Run"
New-Item -ItemType Directory -Force $Evidence | Out-Null

git rev-parse HEAD | Tee-Object "$Evidence/revision.txt"
sonar-scanner -v | Tee-Object "$Evidence/scanner-version.txt"

$sw = [Diagnostics.Stopwatch]::StartNew()
sonar-scanner -Dsonar.projectKey=$ProjectKey -Dsonar.sources=src -Dsonar.sourceEncoding=UTF-8 2>&1 | Tee-Object "$Evidence/scanner.log"
$sw.Stop()
$sw.Elapsed.TotalSeconds | Set-Content "$Evidence/scanner-seconds.txt"

Copy-Item .scannerwork/report-task.txt "$Evidence/report-task.txt"
$ceTaskId = ((Get-Content .scannerwork/report-task.txt) | Where-Object { $_ -like 'ceTaskId=*' }).Split('=')[1]
$headers = @{ Authorization = "Bearer $env:SONAR_TOKEN" }

do {
  $task = Invoke-RestMethod "$env:SONAR_HOST_URL/api/ce/task?id=$ceTaskId" -Headers $headers
  $task | ConvertTo-Json -Depth 8 | Set-Content "$Evidence/ce-task-latest.json"
  if ($task.task.status -in @('SUCCESS','FAILED','CANCELED')) { break }
  Start-Sleep -Seconds 1
} while ($true)

Invoke-RestMethod "$env:SONAR_HOST_URL/api/system/health" -Headers $headers | ConvertTo-Json -Depth 8 | Set-Content "$Evidence/system-health.json"
docker stats --no-stream --format "{{json .}}" sq31-sonarqube sq31-postgres | Set-Content "$Evidence/docker-stats.jsonl"
Write-Host "Evidence: $Evidence ; CE task: $ceTaskId ; status: $($task.task.status)"

Run one warm-up plus at least three measured repetitions on the same revision. After each task reaches SUCCESS, append a row to a worksheet. If a task reaches FAILED or CANCELED, stop the benchmark and diagnose that task; do not average a failed state into a performance number.

5. Derive queue wait and service time instead of lumping them together

The CE task response contains timestamps such as submission, start and execution/completion fields depending on the current endpoint representation. Preserve the raw JSON first. Then calculate queue wait from submit-to-start and processing duration from start-to-finish using fields actually present in your baseline. If the endpoint schema differs, use the API documentation embedded in that exact SonarQube instance; do not invent a field name to make a worksheet work.

# Example derivation after inspecting the raw task JSON.
# Pseudocode — map these names to fields actually returned by your version.
queue_wait_s = (started_at - submitted_at).total_seconds()
ce_service_s = (finished_at - started_at).total_seconds()
end_to_end_server_s = queue_wait_s + ce_service_s

6. Take read-only database, disk and search-adjacent snapshots

We do not query or edit SonarQube’s internal tables. PostgreSQL exposes supported operational statistics that can be read without knowing application schema. Capture the counters before and after a bounded run window. For search, use SonarQube health/system information, container/host disk metrics and SonarQube logs; do not reach into embedded Elasticsearch indices or mutate them directly.

# PostgreSQL operational counters — read-only.
docker exec sq31-postgres psql -U sonar -d sonar -c "
select datname,numbackends,xact_commit,xact_rollback,
       blks_read,blks_hit,temp_files,temp_bytes,
       tup_returned,tup_fetched,tup_inserted,tup_updated,tup_deleted
from pg_stat_database where datname='sonar';"

# Storage and process snapshots.
docker stats --no-stream sq31-sonarqube sq31-postgres
docker exec sq31-sonarqube sh -lc 'df -h /opt/sonarqube/data; du -sh /opt/sonarqube/data /opt/sonarqube/logs'
docker logs --since 10m sq31-sonarqube > evidence/server-window.log 2>&1

7. Change exactly one workload factor: source-file count

Now change only the source workload. Generate 480 files instead of 120, commit the change and tag it. Keep the same server resources, database, scanner family/version, project key, token permissions and host. This makes the comparison interpretable. The new commit is part of the evidence, not noise to hide.

python ../make_fixture.py 480
git add src
git commit -m "workload factor: 480 files"
git tag sq31-large
git rev-parse HEAD

# Repeat one warm-up and at least three measured runs with the same runner.
pwsh ../run-analysis.ps1

Compare scanner time and CE processing time separately. If scanner time grows much more than CE time, the client-side workload changed more than the server service demand. If CE time grows substantially, correlate it with database, disk and process evidence. Do not conclude that “4× files needs 4× RAM”; the measurement describes this fixture on this host, not a universal scaling law.

8. Create a controlled burst to expose queue behavior

Averages hide bursts, so the last experiment submits several independent project analyses close together. Pre-create three local projects sq31-burst-a, sq31-burst-b and sq31-burst-c and grant the local analysis identity Execute Analysis on them. Use the same synthetic source revision, changing only the stable project key per submission. Start them within a few seconds of each other and preserve each ceTaskId.

# Conceptual PowerShell burst. Each working copy has the same source revision
# and its own stable project key. Do not reuse one .scannerwork directory concurrently.
$projects = @('sq31-burst-a','sq31-burst-b','sq31-burst-c')
$jobs = foreach ($p in $projects) {
  Start-Job -ArgumentList $p,$env:SONAR_HOST_URL,$env:SONAR_TOKEN -ScriptBlock {
    param($projectKey,$hostUrl,$token)
    $env:SONAR_HOST_URL=$hostUrl; $env:SONAR_TOKEN=$token
    Set-Location "../copies/$projectKey"
    sonar-scanner -Dsonar.projectKey=$projectKey -Dsonar.sources=src 2>&1
  }
}
$jobs | Wait-Job | Receive-Job

Community Build provides the useful learning constraint here: background work serializes, so closely arriving reports should expose queue wait if they reach the server faster than the Compute Engine can finish them. The scanner jobs may all upload successfully while later CE tasks are still pending. That is exactly why upload success, CE success and end-to-end completion are separate states.

9. Compare your measurements with a reference architecture without normalizing away differences

Create an assumptions-diff table between the current 10M reference architecture and your lab. Your laptop/container results are not expected to match the published host numbers; they are expected to teach you which differences make the comparison invalid as a benchmark.

10. Challenge: choose the layer, not the command

Your large-fixture scanner time doubles, but CE processing time, queue wait, server CPU, database counters and disk latency remain almost unchanged. Which first action is best?

• A. Increase CE workers.
• B. Increase search heap.
• C. Inspect scanner indexing/analyzer time, runner CPU/RAM/cache and effective source scope.
• D. Move PostgreSQL to a larger machine.

11. Cleanup and rollback

First copy the evidence packet outside the lab directory. Revoke/delete the local lab token through the disposable instance UI. Then remove only the named Compose project and volumes created by this lab. Do not use broad Docker prune commands as “cleanup,” because they can delete unrelated development data.

# From the directory that contains this lab's compose.yaml only:
docker compose down -v
cd ..
# Delete sq31-capacity-lab only after confirming evidence was copied out.

# Remove the shell token from the current process.
Remove-Item Env:SONAR_TOKEN -ErrorAction SilentlyContinue

Knowledge check

Official references and version notes

• SonarQube downloads — Current Community Build, commercial Server release train, editions and active LTA.
• Server host requirements — Current disk, memory, CPU, local-storage and production database-host guidance.
• Community Build host requirements — Equivalent Community Build host/search guidance for the mandatory free path.
• Monitoring the instance — Web/CE/search JVMs, read-only JMX MBeans and ComputeEngineTasks/Database signals.
• Reference architecture up to 10 M LOC — Planning anchor and its stated normal-usage assumptions.
• Improving performance — Enterprise+ CE-worker guidance and the requirement to measure external bottlenecks.
• Performance issues — Current troubleshooting starting points for storage, scope, workers and database-related performance.
• Server release notes — 2026.1 runtime/database changes, including JDK 21/25 and PostgreSQL 14–18.
• SonarScanner CLI releases — Current scanner release identity; 8.1.0.6389 is the latest listed release at this chapter recheck.
• Web API — Bearer authentication guidance and the ongoing Web API V2 migration.