Skip to content
This repository was archived by the owner on Jun 26, 2023. It is now read-only.

Use bufio.Reader instead of bufio.Scanner to handle long string in GrepFunc - #23

Merged
babarot merged 1 commit into
babarot:masterfrom
dragon3:use-bufio-reader
Jun 15, 2020
Merged

babarot merged 1 commit into
babarot:masterfrom
dragon3:use-bufio-reader

Conversation

@dragon3

@dragon3 dragon3 commented Jun 15, 2020

Copy link
Copy Markdown
Collaborator

WHAT

This pull request changes using bufio.Reader instead of bufio.Scanner to handle long string in GrepFunc.

WHY

To fix the following error If the text is very long (over: bufio.MaxScanTokenSize):

bufio.Scanner: token too long

@dragon3
dragon3 requested review from babarot and dtan4 June 15, 2020 06:29
@dragon3 dragon3 self-assigned this Jun 15, 2020
text := args[1].AsString()
var matches []string
in := strings.NewReader(text)
scanner := bufio.NewScanner(in)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL: bufio.Scanner has a limitation to read maximum string 🤔 Also, MaxScanTokenSize(64 * 1024) is a const value!

@babarot babarot left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@babarot
babarot merged commit c580daf into babarot:master Jun 15, 2020
@dragon3
dragon3 deleted the use-bufio-reader branch June 15, 2020 09:09
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

⌂ 换网址