lexer

Overview ¶

Package lexer contains a simple lexer for reading an input-string and converting it into a series of tokens.

In terms of syntax we're not very complex, so our lexer only needs to care about simple tokens:

- Comments - Strings - Some simple characters such as "(", ")", "[", "]", "=>", "=", etc. -

We can catch some basic errors in the lexing stage, such as unterminated strings, but the parser is the better place to catch such things.