Skip to content

Report test metrics from a nightly workflow - #3903

Merged
a-TODO-rov merged 2 commits into
redis:mainfrom
bobymicroby:ci/nightly-otel-metrics
Aug 26, 2026
Merged

a-TODO-rov merged 2 commits into
redis:mainfrom
bobymicroby:ci/nightly-otel-metrics

Conversation

@bobymicroby

@bobymicroby bobymicroby commented Aug 26, 2026 •

Copy link
Copy Markdown
Member

Adds a workflow that runs the test suite once a night against the current stable Redis test image and reports per-test durations to Grafana. The exporter reads the JUnit XML reports produced by the build and sends one duration per test; dashboards and alerts watch for slowdowns.

The workflow runs only when the OTEL_CI_VISIBILITY_TOKEN secret is set. Without it the job prints a warning and skips every step, so nothing changes for forks or for this repo until the secret is created.

No existing CI files are touched. The nightly job runs the same steps as the regular test workflow (Java 8, make start version=8.10, make test) in a single configuration, so each test reports exactly one duration per night.

Verified with a full local dry run: 7168 tests parsed from the surefire and failsafe reports, one metric series per test, submitted in a single OTLP request.


Note

Low Risk
Adds an isolated scheduled workflow with no changes to existing CI; risk is limited to optional external metrics export when a secret is configured.

Overview
Adds a new nightly GitHub Actions workflow (nightly-metrics.yaml) that runs once per day (04:30 UTC, staggered from other nightly jobs) and on manual dispatch. It exercises the Lettuce test suite against Redis 8.10 using the same path as regular CI (make start, make test, Java 8).

After tests complete, Surefire and Failsafe JUnit XML is copied into test-results and pushed to Grafana via redis-developer/cae-otel-ci-visibility@v4 (OTLP metrics), with server-version: 8.10 for per-test duration tracking.

The job is gated on OTEL_CI_VISIBILITY_TOKEN: without the secret, a warning is emitted and all other steps are skipped so forks and repos without the secret see no behavior change. Metrics are still reported when tests fail; teardown runs on always() with continue-on-error.

Reviewed by Cursor Bugbot for commit b598e7e. Bugbot is set up for automated code reviews on this repo. Configure here.

bobymicroby and others added 2 commits August 26, 2026 10:07
Adds a workflow that runs the test suite once a night against the
current stable Redis test image and reports per-test durations to
Grafana. The exporter reads the JUnit XML reports produced by the
build and sends one duration per test; dashboards and alerts watch
for slowdowns.

The workflow runs only when the OTEL_CI_VISIBILITY_TOKEN secret is
set. Without it the job prints a warning and skips every step.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit b598e7e. Configure here.

if: ${{ !cancelled() && steps.gate.outputs.run == 'true' }}
run: |
mkdir -p test-results
cp target/surefire-reports/TEST-*.xml target/failsafe-reports/TEST-*.xml test-results/

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Report copy fails if one suite missing

Medium Severity

cp expands both Surefire and Failsafe TEST-*.xml globs in one command. With default bash (no nullglob), an unmatched glob stays literal and cp exits non-zero. When unit tests fail, Maven stops before Failsafe, so that directory is often missing and Collect fails on the path that is supposed to still report metrics.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit b598e7e. Configure here.

@a-TODO-rov
a-TODO-rov merged commit d8d36f3 into redis:main Aug 26, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

⌂ 换网址