Skip to content

Commit dffa370

Browse files
authored
fix(go): tokenize trailing // as comment instead of consuming next line (#1265)
The Go single-line comment rule `//\s+[^\n\r]*` requires a non-newline char after the whitespace, so a `//` at end of line falls through to the rule with greedy `\s+` which consumes the trailing newline plus the next line's indent, swallowing the following line into the comment token. Switches the rules to `//[^\s\n\r][^\n\r]*` (CommentPreproc) and `//[^\n\r]*` (CommentSingle), which keeps build-tag preproc handling intact and matches an empty-body comment correctly. Closes #1264
1 parent 1cf1560 commit dffa370

3 files changed

Lines changed: 30 additions & 2 deletions

File tree

‎lexers/go.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ func goRules() Rules {
3030
"root": {
3131
{`\n`, TextWhitespace, nil},
3232
{`\s+`, TextWhitespace, nil},
33-
{`//[^\s][^\n\r]*`, CommentPreproc, nil},
34-
{`//\s+[^\n\r]*`, CommentSingle, nil},
33+
{`//[^\s\n\r][^\n\r]*`, CommentPreproc, nil},
34+
{`//[^\n\r]*`, CommentSingle, nil},
3535
{`/(\\\n)?[*](.|\n)*?[*](\\\n)?/`, CommentMultiline, nil},
3636
{`(import|package)\b`, KeywordNamespace, nil},
3737
{`(var|func|struct|map|chan|type|interface|const)\b`, KeywordDeclaration, nil},
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
var strings = []string{
2+
"bar", //
3+
"baz", // baz
4+
}
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
[
2+
{"type":"KeywordDeclaration","value":"var"},
3+
{"type":"TextWhitespace","value":" "},
4+
{"type":"NameOther","value":"strings"},
5+
{"type":"TextWhitespace","value":" "},
6+
{"type":"Punctuation","value":"="},
7+
{"type":"TextWhitespace","value":" "},
8+
{"type":"Punctuation","value":"[]"},
9+
{"type":"KeywordType","value":"string"},
10+
{"type":"Punctuation","value":"{"},
11+
{"type":"TextWhitespace","value":"\n\t"},
12+
{"type":"LiteralString","value":"\"bar\""},
13+
{"type":"Punctuation","value":","},
14+
{"type":"TextWhitespace","value":" "},
15+
{"type":"CommentSingle","value":"//"},
16+
{"type":"TextWhitespace","value":"\n\t"},
17+
{"type":"LiteralString","value":"\"baz\""},
18+
{"type":"Punctuation","value":","},
19+
{"type":"TextWhitespace","value":" "},
20+
{"type":"CommentSingle","value":"// baz"},
21+
{"type":"TextWhitespace","value":"\n"},
22+
{"type":"Punctuation","value":"}"},
23+
{"type":"TextWhitespace","value":"\n"}
24+
]

0 commit comments

Comments
 (0)
⌂ 换网址