Skip to content

feat: per-path rules for scan thresholds - #37

Merged
machado144 merged 1 commit into
mainfrom
feat/path-rules
Oct 16, 2026
Merged

machado144 merged 1 commit into
mainfrom
feat/path-rules

Conversation

@machado144

@machado144 machado144 commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What

Per-path rules in .dupehound.yml (spec #36, approved). One threshold rarely fits a repo: test fakes of 20–40 tokens are invisible at 50, and 25 everywhere floods production.

scan:
  min-tokens: 50
rules:                     # first matching rule wins; other files use scan
  - name: tests
    paths: ["**/*_test.py", "**/tests/**"]
    min-tokens: 25
  - name: migrations
    paths: ["**/migrations/versions/*.py"]
    min-tokens: 120
  - name: generated
    paths: ["**/generated/**"]
    skip: true

Before (one threshold, 50): the copied EchoModel fake is not reported. After:

Skipped files : 1 (skip rule)
  #1 type-1 ... 76 tokens 2 instances [cross-file] [rule: scan ≥50]
        src/app.py:10-18, src/other.py:1-9
  #2 type-1 ... 45 tokens 2 instances [cross-file] [rule: tests ≥25]
        tests/test_a.py:1-6, tests/test_b.py:1-6

Decisions

  • Cross-rule clone → highest min-tokens among its files (type-3: highest similarity). Independent of instance order, production is never reported below its own threshold, and test-only copies are still found because the lower threshold runs over the test files alone (the fake above is also in src/app.py, which is at 50, so that copy is left out).
  • One exact pass per distinct threshold, each over the files allowed at that threshold, keeping only the clones judged at it. No rules = the old single pass, so output is unchanged and baseline fingerprints stay stable.
  • Overrides: min-tokens, similarity, skip. CLI --min-tokens/--similarity set the base and rules still apply; --no-rules turns them off.
  • Rule shown in text, md, json (rule, rule_min_tokens), sarif (properties.rule), github annotations.
  • rules: is strict (unknown key, empty/malformed glob, threshold ≤ 0, skip + override, duplicate name → rules[N] (line L): …, exit 2). scan: stays lenient so existing configs keep loading.

Verification

  • TDD: barrier at threshold / one under, cross-rule policy, first match, skip, similarity per rule, low-memory mode, --since scope per pass, baseline hashes unchanged, invalid configs, all five formats.
  • Golden test: binary output without rules and with --no-rules is byte-identical to v0.2.0's (integration/testdata/rules.norules.*.golden, recorded with the main binary).
  • Go net (378 files): no rules byte-identical to main; with a tests rule at 25 the 225 clones touching production are identical (hash + locations), 6.2 s → 10.2 s, peak RSS +3%.
  • go test -race ./..., golangci-lint, gofumpt, structlint, govulncheck clean; self-scan reports 0 clones.

Backward compatibility (no rules:)

v0.2.0 and this branch, built from source, run against dupehound, structlint, loadout, tacomex-8bit-shop, switchyard, qorder-app, replayraccoon and quasar (Go, TS, Python, Rust), each with its own .dupehound.yml:

  • json, text, sarif, md and github output byte-identical, same exit codes; also with --dead-code, --scope function, --git-churn, --since, --staged, low-memory mode (--max-memory 120M), and empty, comment-only and unknown-key configs.
  • --write-baseline files identical; a v0.2.0 baseline suppresses the same clones (same known/new counts) with the new binary.
  • gauntlet check output identical. qorder-app (804 files, 134k lines), 7 runs each: 7.22 s vs 7.30 s mean, same peak RSS.

Refs #36

* rules: in .dupehound.yml overrides min-tokens/similarity or skips files by glob; first match wins
* cross-rule clones are judged by the highest min-tokens among their files
* one exact pass per distinct threshold; no rules = byte-identical output
* rule shown in every format; --no-rules flag; validated with rule and line in errors

Refs #36

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

✅ PR title follows the required format

Current title: feat: per-path rules for scan thresholds

@machado144
machado144 marked this pull request as ready for review October 16, 2026 11:24
@machado144
machado144 merged commit e7730b0 into main Oct 16, 2026
8 checks passed
@machado144
machado144 deleted the feat/path-rules branch October 16, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

⌂ 换网址