Skip to content

feat: per-file duplication stats and global duplication percentage #6

Closed

Description

@machado144

Summary

A project-level duplication percentage and per-file breakdown transforms dupehound from a "here are the clones" tool into a genuine code quality gate. Teams can track improvement over time, set thresholds, and quickly identify the most duplicated files — exactly the information needed to prioritise refactoring.

New report fields

Global (domain.Report)

{
  "scanned_files": 42,
  "total_clones": 7,
  "duplicate_lines": 183,
  "total_lines": 2410,
  "duplication_pct": 7.6
}

Per-file breakdown (domain.FileStats)

"file_stats": [
  { "file": "services/handler.go", "total_lines": 312, "duplicate_lines": 88, "duplication_pct": 28.2 },
  { "file": "services/processor.go", "total_lines": 204, "duplicate_lines": 41, "duplication_pct": 20.1 }
]

File stats sorted descending by duplication_pct, only files with > 0 duplicates included.

New flag

--min-duplication float   fail (exit 1) if global duplication_pct exceeds this value (default: 0 = disabled)

Combined with issue #1 exit codes:

  • Clones found → exit 1 (unless --exit-zero)
  • --min-duplication 5.0 and project is at 7.6% → exit 1

Text output changes

Scanned 42 files · 2410 lines · 7 clones · 183 duplicate lines (7.6%)

Top duplicated files:
  28.2%  services/handler.go      (88/312 lines)
  20.1%  services/processor.go    (41/204 lines)

Clone #1  type-2  similarity: 1.00  ...

Implementation checklist

Domain

  • domain/types.go: add TotalLines int, DuplicationPct float64 to Report
  • domain/types.go: add FileStats []FileStats to Report
  • domain/types.go: define FileStats struct { File string; TotalLines int; DuplicateLines int; DuplicationPct float64 }

Scanner

  • services/scanner.go: count TotalLines per file (len of RawLines) and sum globally
  • services/scanner.go: compute DuplicationPct = DuplicateLines / TotalLines * 100
  • services/scanner.go: build FileStats slice after clone detection: for each file, count lines touched by any clone instance
  • services/scanner.go: sort FileStats by DuplicationPct descending, filter to pct > 0

Flags & action

  • flags.go: add --min-duplication flag (float64, default 0)
  • services/scanner.go: add MinDuplication float64 to ScanOptions
  • actions/scan.go: after scan, if MinDuplication > 0 && report.DuplicationPct > MinDuplication → return ErrClonesFound (or a new ErrThresholdExceeded)

Reporter

  • services/reporter.go: text header line updated to include duplicate lines (X.X%)
  • services/reporter.go: text output shows "Top duplicated files" section (only when len(FileStats) > 0)
  • services/reporter.go: JSON marshalling picks up new fields automatically (verify field names use snake_case json tags)
  • services/reporter.go: SARIF run-level properties includes duplicationPct and duplicateLines

Tests

  • services/scanner_test.go: report.TotalLines > 0 after scanning real files
  • services/scanner_test.go: report.DuplicationPct is in range (0, 100] when clones exist
  • services/scanner_test.go: report.FileStats is non-empty and sorted descending by pct
  • services/scanner_test.go: --min-duplication threshold below actual pct → error returned
  • services/scanner_test.go: --min-duplication threshold above actual pct → no error
  • integration/cli_test.go: JSON output includes total_lines and duplication_pct
  • integration/cli_test.go: text output contains the duplication percentage
  • integration/cli_test.go: --min-duplication 1 on a codebase with duplicates → exit 1
  • integration/cli_test.go: --min-duplication 100 on a codebase with duplicates → exit 0

Build & quality

  • make test passes
  • lefthook pre-commit passes
  • No regressions

Activity

  1. added 3 commits that reference this issue on Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      ⌂ 换网址