Nice post. A while back, I wrote a package that does all the heavy lifting for you (file / position tracking, next/backup, etc.): https://github.com/db47h/lex

Thanks for reading! Your lex package looks really well thought out. I like how the design resembles a DFA!

Thanks! Like I mention in the docs, it's heavily inspired by Rob Pike's talk on using functions values as state.

Thanks for this blog Aaron. I recently wrote a JSONPath lexer in Go and found two other resources very helpful: Rob Pike's talk "Lexical Scanning in Go" and Jack Crenshaw's classic How to build a compiler.

Thanks for sharing! Rob Pike's talk is one of the resources that I came across when I was getting started; it's a great presentation! I've never read Jack Crenshaw's work but I will check it out!

Feel free to link your JSONPath lexer, I would love to check it out!

Er, I did link it. :-)

Oops, I totally missed that! I should look into how to change the link styling in the comments as it can be hard to notice sometimes.

Anyways, I like the use of a stack in the lexer! The descriptive function names makes the code easy to read. Nice work!

Good idea about the link styling. Oh and did you notice the unit tests and fuzz testing? :-)

Yeah, the fuzz testing is super cool! It's something I have essentially no experience with, but would love to learn more about it.

It looks like you have really good test coverage!

I tried fuzz testing for the first time last week after 38 years writing software. It turned out to be really easy.

On coverage, I don't necessarily aim for 100% as that's pretty meaningless, but I do use coverage to spot missing tests.

Awesome post!! Really great stuff, very clean and clear with concise examples. Do you have plans to write about the next step, i.e. parsers?

Thanks for the kind words :) I'm planning on publishing a post on writing a parser in the next week or so!

Very cool! I look forward to it.

Hey Justin! I've decided not to write a post on writing a parser. Mostly because there are some very good resources already out there, and I'm not too sure that I have much to add to them. This section of Crafting Interpreters is fantastic and I would highly recommend checking it out!

I am still planning on publishing a post this weekend though!

Aw what a shame, I really enjoy your writing style and easy breakdown of these concepts. Well I will check out that resource, thanks for the reccomendation.

I look forward to your next post!

If you have any questions, feel free to send me an email or reach out on Twitter!

Nice post. But I think use compound function is better. https://github.com/acekingk...
use `repeate ,concat, select` can work well.

Could you please continue write. Could you tell more about AST with source example. Thanks.