CODEWITHSUNDEEP
parsingcompilationcompiler-constructionlanguage-agnostic
Dean
Dean

Reputation: 6958

Fast lexing during edit-time

I'm wondering what techniques are employed to fast lex a huge source file (e.g. C++) in popular text editors.

Re-lexing an entire document each time I edit the file (i.e. I add some characters) could quickly become unfeasible.. but those characters might change everything (towards the top, towards the bottom or both) into the source file, as it is the case when inserting

/*

or

\*

halfway through the huge file.

Is there a standard approach to runtime lexing?

Upvotes: 2

Views: 128

Answers (1)

Ira Baxter
Ira Baxter

Reputation: 95420

I'd assume the same techniques that are used to lex source files by compilers.

A good FSA-based lexer (or a hand-written one) spends only a few instructions per character. Assume a klunky machine that takes 5 nS to execute an instruction (way slow WRT modern workstations). If you have a million-character buffer (that's about 300,000 lines of 30 characters), and it takes 10 instructions to process each, the total lexing time is 50 million nS = .05 second.

Why do you think this is a problem?

Upvotes: 2

Related Questions