Repository navigation
Report test metrics from a nightly workflow - #3903
Merged
Merged
Conversation
Adds a workflow that runs the test suite once a night against the current stable Redis test image and reports per-test durations to Grafana. The exporter reads the JUnit XML reports produced by the build and sends one duration per test; dashboards and alerts watch for slowdowns. The workflow runs only when the OTEL_CI_VISIBILITY_TOKEN secret is set. Without it the job prints a warning and skips every step.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Reviewed by Cursor Bugbot for commit b598e7e. Configure here.
| if: ${{ !cancelled() && steps.gate.outputs.run == 'true' }} | ||
| run: | | ||
| mkdir -p test-results | ||
| cp target/surefire-reports/TEST-*.xml target/failsafe-reports/TEST-*.xml test-results/ |
There was a problem hiding this comment.
Report copy fails if one suite missing
Medium Severity
cp expands both Surefire and Failsafe TEST-*.xml globs in one command. With default bash (no nullglob), an unmatched glob stays literal and cp exits non-zero. When unit tests fail, Maven stops before Failsafe, so that directory is often missing and Collect fails on the path that is supposed to still report metrics.
Reviewed by Cursor Bugbot for commit b598e7e. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Adds a workflow that runs the test suite once a night against the current stable Redis test image and reports per-test durations to Grafana. The exporter reads the JUnit XML reports produced by the build and sends one duration per test; dashboards and alerts watch for slowdowns.
The workflow runs only when the OTEL_CI_VISIBILITY_TOKEN secret is set. Without it the job prints a warning and skips every step, so nothing changes for forks or for this repo until the secret is created.
No existing CI files are touched. The nightly job runs the same steps as the regular test workflow (Java 8, make start version=8.10, make test) in a single configuration, so each test reports exactly one duration per night.
Verified with a full local dry run: 7168 tests parsed from the surefire and failsafe reports, one metric series per test, submitted in a single OTLP request.
Note
Low Risk
Adds an isolated scheduled workflow with no changes to existing CI; risk is limited to optional external metrics export when a secret is configured.
Overview
Adds a new nightly GitHub Actions workflow (
nightly-metrics.yaml) that runs once per day (04:30 UTC, staggered from other nightly jobs) and on manual dispatch. It exercises the Lettuce test suite against Redis 8.10 using the same path as regular CI (make start,make test, Java 8).After tests complete, Surefire and Failsafe JUnit XML is copied into
test-resultsand pushed to Grafana viaredis-developer/cae-otel-ci-visibility@v4(OTLP metrics), withserver-version: 8.10for per-test duration tracking.The job is gated on
OTEL_CI_VISIBILITY_TOKEN: without the secret, a warning is emitted and all other steps are skipped so forks and repos without the secret see no behavior change. Metrics are still reported when tests fail; teardown runs onalways()withcontinue-on-error.Reviewed by Cursor Bugbot for commit b598e7e. Bugbot is set up for automated code reviews on this repo. Configure here.