Repository navigation
feat(#6): per-file duplication stats and global duplication percentage - #12
Conversation
…d flag - Add TotalLines, DuplicationPct, and FileStats to Report - FileStats includes per-file total_lines, duplicate_lines, and duplication_pct, sorted by percentage descending - Add --min-duplication flag: fail (exit 1) when global duplication percentage exceeds the threshold - Update text output with percentage and hotspots sorted by dup % - Update markdown output with dup % columns in hotspots table - JSON output includes all new fields automatically
StructLint — All checks passed48 rules validated against
|
There was a problem hiding this comment.
Core Changes
This PR introduces significant enhancements to dupehound's reporting capabilities and adds a new quality gate feature:
- Per-file Duplication Statistics: A new
FileStatsstruct is added to thedomain.Report, providingtotal_lines,duplicate_lines, andduplication_pctfor each scanned file. These statistics are calculated inservices/scanner.goand sorted by duplication percentage for hotspot reporting. - Global Duplication Percentage: The
domain.Reportnow includesTotalLinesandDuplicationPctfields, summarizing the overall code duplication across the entire scan. - Duplication Threshold Quality Gate: A new
--min-duplicationCLI flag (and corresponding logic inactions/scan.go) allows users to specify a maximum acceptable global duplication percentage. If the scan'sDuplicationPctexceeds this threshold, the tool will exit with code 1 (similar to finding clones), enabling its use in CI/CD pipelines as a quality gate. - Enhanced Reporting: The
textandmarkdownoutput formats inservices/reporter.goare updated to display the new global and per-file duplication metrics, including percentages and improved hotspot tables.
Concerns (if any)
None. The changes are well-implemented and address the stated goals effectively.
Verdict
Approve
Code review performed by GEMINI - gemini-2.5-flash.
There was a problem hiding this comment.
Core Changes
This PR introduces significant enhancements to dupehound's reporting and CLI functionality:
- Per-file Duplication Statistics: The
domain.Reportnow includesFileStatswhich providestotal_lines,duplicate_lines, andduplication_pctfor each scanned file, sorted by duplication percentage. This allows users to identify 'hotspots' of duplication more easily. - Global Duplication Percentage: The overall
total_linesandduplication_pctare now calculated and included in thedomain.Reportand displayed in the text and Markdown output formats. - Duplication Threshold Quality Gate: A new
--min-duplicationCLI flag has been added. If set to a value greater than 0, the tool will exit with code 1 (indicating failure) if the global duplication percentage exceeds this threshold. This enables dupehound to be used as a quality gate in CI/CD pipelines. - Improved Output Formats: Both text and Markdown reports have been updated to display the new total lines and duplication percentage metrics, and the 'hotspots' section now leverages the
FileStatsto show per-file duplication percentages.
Concerns
- Missing Integration Tests for Quality Gate: The PR description explicitly states that integration tests for the new
--min-duplicationthreshold feature are "to be added in follow-up". This is a critical new feature that directly affects the tool's exit behavior and thus its usability as a quality gate. Without dedicated integration tests, there's a risk of regressions or incorrect behavior in real-world scenarios. These tests should ideally be part of this PR to ensure the feature is fully validated before merging.
Verdict
Request Changes
Code review performed by GEMINI - gemini-2.5-flash.
…eport fields - Threshold exceeds → exit 1 - Threshold below → exit 0 - Threshold disabled (default 0) → exit 0 with --exit-zero - JSON output includes total_lines, duplication_pct, file_stats - Text output includes duplication percentage and Total lines
dupehound scan results
Hotspots
Top clones (by impact)#1
|
There was a problem hiding this comment.
Core Changes
This PR introduces significant enhancements to dupehound's reporting and CLI functionality:
- Duplication Metrics: The
Reportstructure now includesTotalLinesandDuplicationPct(global duplication percentage). A newFileStatsstruct has been added to provide per-file duplication metrics, includingTotalLines,DuplicateLines, andDuplicationPctfor each file, sorted by duplication percentage. - Quality Gate Flag: A new
--min-duplicationCLI flag (and correspondingErrThresholdExceedederror) allows users to set a threshold for global duplication percentage. If the calculated duplication exceeds this value, the tool will exit with code 1, enabling its use as a quality gate in CI/CD pipelines. - Output Enhancements: Text and Markdown reports have been updated to display the new global and per-file duplication statistics. The 'Hotspots' section now leverages
FileStatsand sorts files by duplication percentage. JSON output automatically includes all new fields.
Concerns
None. The changes are well-tested and logically sound.
Verdict
Approve
Code review performed by GEMINI - gemini-2.5-flash.
Summary
Closes #6
FileStatswithtotal_lines,duplicate_lines,duplication_pctsorted by percentage descendingtotal_linesandduplication_pctadded toReport--min-duplicationflag: fail (exit 1) when global duplication % exceeds threshold — enables use as a quality gateTest plan
--min-duplication 5.0 --exit-zeroon simplycodes-web → exit 1--min-duplication 50.0 --exit-zero→ exit 0total_lines,duplication_pct,file_stats