Skip to content

Commit 385f3e2

Browse files
fredbiclaude
andcommitted
fix(builders): don't fail a scan over a declaration that cannot be read
A package served from compiled export data has types and no syntax, so asking for the declaration of one of its types answers nothing. Four builder sites treated that as fatal and ended the scan: a `time.Duration` field, an `io.Writer`, a `reflect.Type`, and the schema builder's own entry point. The whole document was lost over prose that was never going to be there. Those sites degrade now. The type is rendered from what the type-checker knows — which is everything except the comments — and a scan.sourceless-type warning says which type it was, where the field is, and what was given up. Losing a description a dependency happened to carry is a small, stated cost; losing the spec is not. The author's own annotations always win, so a `swagger:description` override is the answer where the prose matters. The asymmetry is closed too: a definition that goes missing altogether — rather than merely thinning — warns on the same code, so nothing disappears in silence. What stays unrendered is a model declared in a dependency that carries no annotation anywhere: the marker scan passes over it, correctly, and the definition collapses to a bare `x-go-name`. Reading back every dependency outside the standard library instead brings the definition back EMPTY and without the warning, because struct fields are located by position and an export-data object and a separately parsed file are two token.File entries for one filename. Closing it needs a name-based field bridge in the schema builder, so it is written down where it would be retried, and CompiledDependencies stays opt-in and experimental until it is. Also records what the standard-library recognizer set deliberately leaves out. A recognizer asserts that a type means one thing on the wire wherever it appears, so one may only be added where that holds for every use. `time.Duration` is the case that invites it: a count of nanoseconds, where whether an API means nanoseconds, seconds or an ISO 8601 string is the author's decision rather than a property of the type. That is what strfmt.Duration exists for, and teaching the set to guess would take the choice away. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
1 parent 0ce3785 commit 385f3e2

16 files changed

Lines changed: 400 additions & 46 deletions

File tree

‎internal/builders/common/builder.go‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,50 @@ type Builder struct {
3939
blockCache map[*ast.CommentGroup][]grammar.Block
4040
}
4141

42+
// SourcelessFallback reports whether a missing declaration should be rendered from the type alone
43+
// rather than failing the scan, and says so when it should.
44+
//
45+
// A declaration lookup comes back empty for two unrelated reasons, and only one of them is a fault.
46+
// The load may have deliberately not read that package — the standard library under a WebAssembly
47+
// guest, any unannotated dependency under compiled dependencies — in which case the type is complete
48+
// and only what its author wrote about it is missing. Or the graph is broken, in which case failing
49+
// is right and a thinner document would hide it.
50+
//
51+
// So the strict sites ask here first. They used to fail on both, which meant one field of a type
52+
// nobody can render anyway — time.Duration, io.Writer, reflect.Type — took a whole document with it.
53+
// Worse, they failed inconsistently: the same absent declaration degrades quietly at the soft sites
54+
// two arms away, so the outcome depended on which position the type happened to appear in rather
55+
// than on anything about the type.
56+
//
57+
// The remedy offered is deliberately not "recognize more standard-library types". A recognizer
58+
// asserts a wire form for every use of a type, and these are precisely the ones where no such form
59+
// exists to assert — which is what strfmt.Duration is for. What the author loses is a doc comment
60+
// they rarely wanted in their API, and swagger:description puts it back where they can see it.
61+
func (s *Builder) SourcelessFallback(obj *types.TypeName) bool {
62+
if obj == nil || obj.Pkg() == nil {
63+
return false
64+
}
65+
66+
if _, sourceless := s.Ctx.SourcelessPackage(obj.Pkg().Path()); !sourceless {
67+
return false
68+
}
69+
70+
// Located at the declaration that consumed the type, not at the type itself: the author can act on
71+
// their own source, and the position of a standard-library declaration they cannot read is no help.
72+
var at token.Position
73+
if s.Decl != nil {
74+
at = s.Ctx.PosOf(s.Decl.Pos())
75+
}
76+
77+
s.RecordDiagnostic(grammar.Warnf(at, grammar.CodeSourcelessType,
78+
"%s.%s is rendered from its type alone, because its declaring package was loaded without source: "+
79+
"anything the declaration said about it — its doc comment, any swagger annotation — is not in "+
80+
"the spec. Write a swagger:description where it is used if it matters.",
81+
obj.Pkg().Path(), obj.Name()))
82+
83+
return true
84+
}
85+
4286
// New builds a [Builder] bound to ctx and decl.
4387
//
4488
// The blockCache is pre-allocated empty.

‎internal/builders/parameters/parameters.go‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -362,6 +362,14 @@ func (p *Builder) buildNamedField(ftpe *types.Named, typable ifaces.SwaggerTypab
362362

363363
decl, found := p.Ctx.DeclForType(o.Type())
364364
if !found {
365+
// No declaration to read, but the type is complete: render what it is rather than what it was
366+
// called. time.Duration lands here and comes out as its underlying int64, which is the same answer
367+
// a readable declaration produces for a parameter — the declaration was only ever going to add
368+
// prose.
369+
if p.SourcelessFallback(o) {
370+
return schema.Delegate(p.Builder, schema.WithType(ftpe.Underlying(), typable))
371+
}
372+
365373
return fmt.Errorf("unable to find package and source file for: %s: %w", ftpe.String(), ErrParameters)
366374
}
367375

‎internal/builders/responses/responses.go‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -402,6 +402,12 @@ func (r *Builder) buildNamedField(ftpe *types.Named, typable ifaces.SwaggerTypab
402402

403403
decl, found := r.Ctx.DeclForType(o.Type())
404404
if !found {
405+
// See the parameters builder's twin: the type is complete even when its declaration is not
406+
// readable, so it is rendered from its underlying shape rather than costing the document.
407+
if r.SourcelessFallback(o) {
408+
return schema.Delegate(r.Builder, schema.WithType(ftpe.Underlying(), typable))
409+
}
410+
405411
return fmt.Errorf("unable to find package and source file for: %s: %w", ftpe.String(), ErrResponses)
406412
}
407413

‎internal/builders/schema/README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,23 @@ Three layers, all in `special_types.go`:
167167
identity-based and cannot misfire on user types, so this helper is
168168
**called uniformly at every site** that handles a `*types.TypeName`.
169169

170+
**What the set deliberately excludes.** A recognizer asserts that a
171+
type means one thing on the wire wherever it appears, so one may only
172+
be added where that holds for every use. `time.Duration` is the
173+
standing counter-example: it counts nanoseconds, and whether an API
174+
means nanoseconds, seconds or an ISO 8601 string is the author's
175+
decision rather than a property of the type — which is why go-openapi
176+
publishes `strfmt.Duration`, carrying a stated wire format, instead of
177+
teaching this set to guess at one. `io.Writer` stays out of the
178+
opaque-stream table for the same reason, see
179+
[§opaque-streams](#opaque-streams).
180+
181+
A consumed type with no recognizer is therefore **reported, not
182+
resolved by assumption**: where its declaring package was served
183+
without source, the scanner says the declaration could not be read,
184+
and the author states what they meant with a strfmt type or an
185+
annotation.
186+
170187
The two UUID recognizers are a **certain/guessed pair**, and the
171188
distinction is the whole reason both exist:
172189

‎internal/builders/schema/embedded.go‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,12 @@ func (s *Builder) buildNamedEmbedded(tpe *types.Named, schema *oaispec.Schema, n
8686
case *types.Struct:
8787
decl, found := s.Ctx.GetModel(tpe.Obj().Pkg().Path(), tpe.Obj().Name())
8888
if !found {
89+
// Nothing to promote from a struct whose declaration cannot be read: the embedding type keeps
90+
// its own fields and loses the embedded ones, which is a smaller object rather than a wrong one.
91+
if s.SourcelessFallback(tpe.Obj()) {
92+
return nil
93+
}
94+
8995
return missingSource(tpe)
9096
}
9197
s.Ctx.AddDiscoveredModel(decl)
@@ -105,6 +111,12 @@ func (s *Builder) buildNamedEmbedded(tpe *types.Named, schema *oaispec.Schema, n
105111
resolvers.MustNotBeABuiltinType(o)
106112
decl, found := s.Ctx.GetModel(o.Pkg().Path(), o.Name())
107113
if !found {
114+
// As above: an embedded interface whose declaration is unreadable promotes no method-derived
115+
// properties rather than taking the document with it.
116+
if s.SourcelessFallback(o) {
117+
return nil
118+
}
119+
108120
return missingSource(tpe)
109121
}
110122
s.Ctx.AddDiscoveredModel(decl)

‎internal/builders/schema/ref.go‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,21 @@ func (s *Builder) resolveRefOr(tio *types.TypeName, tgt ifaces.SwaggerTypable, o
1818
if decl, ok := s.Ctx.GetModel(tio.Pkg().Path(), tio.Name()); ok {
1919
return s.MakeRef(decl, tgt)
2020
}
21+
22+
// A miss with nothing to fall back on is the whole definition going missing — the property comes out
23+
// bare, carrying its name and nothing else. Worth more than a Hint when the cause is a package the
24+
// load did not read: under an ordinary scan this same type is discovered and $ref'd, so unlike the
25+
// thinner-schema cases the difference here is a definition that exists or does not.
26+
//
27+
// Only the no-fallback arm asks. Where orElse renders the type structurally the definition is not
28+
// lost, and the miss is the ordinary "this type is not a model, inline it" outcome that fires on
29+
// every scan.
2130
if orElse == nil {
31+
s.SourcelessFallback(tio)
32+
2233
return nil
2334
}
35+
2436
return orElse()
2537
}
2638

@@ -32,5 +44,12 @@ func (s *Builder) resolveRefOrErr(tio *types.TypeName, tgt ifaces.SwaggerTypable
3244
if decl, ok := s.Ctx.GetModel(tio.Pkg().Path(), tio.Name()); ok {
3345
return s.MakeRef(decl, tgt)
3446
}
47+
// An interface with no declaration to read renders as the open schema, which is what its structural
48+
// form says on its own: any object satisfies it. Strictly less than the $ref a readable declaration
49+
// would have produced, and strictly more useful than losing the document.
50+
if s.SourcelessFallback(tio) {
51+
return nil
52+
}
53+
3554
return missingSource(errTpe)
3655
}

‎internal/builders/schema/schema.go‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -336,10 +336,11 @@ func (s *Builder) buildDeclAlias(tpe *types.Alias, target ifaces.SwaggerTypable)
336336
// shape on the alias's OWN definition rather than chaining a $ref to a separately-built
337337
// definition.
338338
//
339-
// This matches what the sibling *types.Alias branch already does, what TransparentAliases mode
340-
// already does for stdlib decls, and eliminates the Q30 noise from stdlib godocs landing on the
341-
// chain targets (Time / RawMessage) — those targets no longer exist in the spec when the alias
342-
// inlines the canonical shape directly.
339+
// This matches what the sibling *types.Alias branch already does, and what TransparentAliases
340+
// mode already does for stdlib decls. It also keeps the standard library's prose out of the
341+
// document: chaining published a definition for the target (Time / RawMessage) carrying whatever
342+
// godoc that stdlib declaration happened to have, and the target does not exist at all once the
343+
// alias inlines the canonical shape directly.
343344
//
344345
// For predeclared `error`, this is also the only safe path: it has no package, so the GetModel
345346
// lookup below would nil-panic on Pkg().Path().

‎internal/builders/schema/special_types.go‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,18 @@ const (
9292
// fail — and for the predeclared `error` it always fails, since a predeclared object has no package
9393
// and therefore no declaring source to find.
9494
//
95+
// What this set does NOT hold is as much a decision as what it does. A recognizer asserts that a type
96+
// means one thing on the wire wherever it appears, and it may only be added where that is true of
97+
// every use. time.Duration is the standing counter-example: it is a count of nanoseconds, and whether
98+
// an API means nanoseconds, seconds or an ISO 8601 string is the author's choice rather than a
99+
// property of the type — which is why go-openapi publishes strfmt.Duration, carrying a stated wire
100+
// format, instead of teaching this set to guess at one. io.Writer is kept out of the opaque-stream
101+
// table for the same reason.
102+
//
103+
// So a consumed type with no recognizer is reported rather than resolved by assumption: the scanner
104+
// says its declaration could not be read, and the author says what they meant with a strfmt type or
105+
// an annotation.
106+
//
95107
// # Details
96108
//
97109
// See [§special-types](./README.md#special-types).

‎internal/integration/loader_agreement_test.go‎

Lines changed: 54 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -86,8 +86,9 @@ func abConfigs(tb testing.TB) []abConfig {
8686
apply: func(o *codescan.Options) { o.ToolchainFreeLoader = true },
8787
},
8888
{
89-
// Axis B. go/packages takes dependency types from `go list -export`, wholesale: there is
90-
// no per-dependency choice on this route, so a dependency's own annotations are gone.
89+
// Axis B. go/packages takes dependency types from `go list -export` wholesale — a LoadMode
90+
// is one value for the whole load — and the annotated ones are read back afterwards, which
91+
// is how it reaches the per-dependency outcome the toolchain-free route decides during it.
9192
name: "compiled-dependencies",
9293
apply: func(o *codescan.Options) { o.CompiledDependencies = true },
9394
},
@@ -179,43 +180,63 @@ func abCuratedTargets() []string {
179180
// with why. Everything not listed must agree exactly. Keys are "<config>|<target>"; regenerate the
180181
// table with CODESCAN_AB_REPORT=1.
181182
//
182-
// The entries fall into three families, and the split is the finding.
183+
// Two families remain, and what left the table says as much as what is still on it.
183184
//
184-
// 1. THE DECLARATION CONTRACT — the scan does not degrade, it FAILS. A builder asks for the
185-
// declaration of a type whose declaring package has no syntax, and errors out. Both export-data
186-
// configurations hit it, because both produce a package with types and no AST. This is the
187-
// state on-demand materialisation exists to fill in, and it is the reason a decision has to be
188-
// taken about what an absent syntax half means before laziness spreads any further.
185+
// CLOSED — a dependency's own annotations. `swagger:strfmt` marks live in strfmt's source, and
186+
// export data carries types, not comments. The go/packages route used to take every dependency
187+
// from `go list -export` wholesale; it now reads back the source of the ones whose files carry the
188+
// marker, so both routes keep those marks.
189189
//
190-
// 2. A DEPENDENCY'S OWN ANNOTATIONS ARE LOST — CLOSED. `swagger:strfmt` marks live in strfmt's
191-
// source, and export data carries types, not comments. The go/packages route used to take every
192-
// dependency from `go list -export` wholesale; it now reads back the source of the ones whose
193-
// files carry the marker, so both routes keep those marks and these targets agree.
190+
// CLOSED — a stdlib type whose rendering its declaration never decided. `time.Duration` in a
191+
// response header comes out `integer/int64` from its underlying type, byte-identical to the
192+
// baseline: the declaration was only ever going to add prose. That the fallback lands exactly on
193+
// the full-source answer here is the evidence it is the right fallback.
194194
//
195-
// 3. A DEPENDENCY-DECLARED MODEL COLLAPSES — OPEN, and not for family 2's reason after all. The
196-
// model is declared in `scan-repo-boundary/makeplans`, which carries no annotation anywhere in
197-
// its source: the marker scan passes over it, correctly, and the definition still collapses to a
198-
// bare `x-go-name`. Both routes now behave the same way, so this is what the export-data trade
199-
// costs rather than a difference between them. Closing it would mean reading a dependency
200-
// because a scan REACHES it, which is a different rule from reading one because it has something
201-
// to say.
195+
// 1. A STDLIB TYPE LOSES ITS PROSE. The scan no longer FAILS on these — that was the defect worth
196+
// fixing, since one unreadable field took a whole document with it — but the schema is thinner:
197+
// an unreadable interface renders as the open schema rather than a definition. What is lost is
198+
// the declaration's doc comment, which for a stdlib type means the standard library's own prose
199+
// appearing in someone's API. Each raises a scan.sourceless-type Warning naming the type, and
200+
// swagger:description puts a better answer back. Kept as a divergence because it IS one, not
201+
// because it is a fault.
202202
//
203-
// Family 1 closes when the declaration contract does. Either way the assertion below flips to a
204-
// failure, which is how these announce that they are stale.
203+
// 2. A DEPENDENCY-DECLARED MODEL COLLAPSES. The model is declared in `scan-repo-boundary/makeplans`,
204+
// which carries no annotation anywhere in its source: the marker scan passes over it and the
205+
// definition collapses to a bare `x-go-name`. Only compiled-dependencies is affected — the
206+
// toolchain-free route's bundle covers the standard library, so everything else misses it and
207+
// falls through to source, makeplans included.
208+
//
209+
// DO NOT CLOSE THIS BY WIDENING THE READ-BACK RULE. Tried 2026-08-08: read back every dependency
210+
// outside the standard library, annotated or not, which is exactly the line the other route
211+
// draws. The definition comes back, and comes back EMPTY with no warning — worse than its
212+
// absence, since GetModel then succeeds and the warning that named it stops firing.
213+
//
214+
// Struct fields are located by POSITION (structFieldCarrier -> resolvers.FindASTField(decl.File(),
215+
// fld.Pos())). The *types.Var comes from export data and the AST from our own parse: two
216+
// token.File entries for one filename, different bases, positions that never match, every field
217+
// silently skipped. Parse-and-bridge carries TYPE-LEVEL comments — all §5.2 needed for strfmt's
218+
// marks — and cannot carry field-level correspondence, because a struct field is not in package
219+
// scope for bridgeDefs to map.
220+
//
221+
// Closing it needs a name-based field bridge plus a fallback in the field walk: a schema-builder
222+
// change with its own witness, not a loader tweak. Parked, and it is the prerequisite for
223+
// CompiledDependencies ever defaulting on.
224+
//
225+
// Either way the assertion below flips to a failure when one of these closes, which is how they
226+
// announce that they are stale.
205227
func abExpected() map[string]string {
206228
return map[string]string{
207-
// (1) the declaration contract, stdlib — both routes.
208-
"compiled-dependencies|./bugs/2248/...": "builders/responses asks for the declaration of time.Duration, " +
209-
"which has types and no syntax: `unable to find package and source file for: time.Duration`",
210-
"export-data|./bugs/2248/...": "same as compiled-dependencies: time.Duration has no declaration to find",
211-
"compiled-dependencies|./enhancements/opaque-streams/...": "builders/schema asks for the declaration of " +
212-
"io.Writer: `can't find source file for type: interface{Write(p []byte) (n int, err error)}`",
213-
"export-data|./enhancements/opaque-streams/...": "same as compiled-dependencies: io.Writer has no declaration to find",
214-
"compiled-dependencies|./goparsing/go123/...": "builders/schema asks for the declaration of reflect.Type, " +
215-
"a stdlib interface with no syntax behind it",
216-
"export-data|./goparsing/go123/...": "same as compiled-dependencies: reflect.Type has no declaration to find",
217-
218-
// (3) the model itself is declared in a dependency that says nothing about itself.
229+
// (1) a stdlib type with no declaration to read and no identity recognizer to answer for it.
230+
// Degrades and warns; used to fail the scan.
231+
"compiled-dependencies|./enhancements/opaque-streams/...": "io.Writer renders as the open schema " +
232+
"instead of a definition whose description is io.Writer's own method godoc",
233+
"export-data|./enhancements/opaque-streams/...": "same as compiled-dependencies: io.Writer has no declaration to read",
234+
"compiled-dependencies|./goparsing/go123/...": "reflect.Type renders as the open schema, same cause: " +
235+
"a stdlib interface with no syntax behind it; reflect.Value loses its definition outright",
236+
"export-data|./goparsing/go123/...": "same as compiled-dependencies: reflect.Type and reflect.Value " +
237+
"have no declaration to read",
238+
239+
// (2) the model itself is declared in a dependency that says nothing about itself.
219240
"compiled-dependencies|./goparsing/bookings/...": "the Booking model is declared in " +
220241
"scan-repo-boundary/makeplans, which carries no annotation, so the marker scan does not read it " +
221242
"back and the definition collapses to a bare `x-go-name: Booking`",

0 commit comments

Comments
 (0)
⌂ 换网址