CODEWITHSUNDEEP
pythonlexical-analysis
Thor Correia
Thor Correia

Reputation: 1588

I need a way to lexically split my strings

For example, if I'm given this string

"asdf bob(1, 2, "a")"

I would like it to be split as

["asdf", "bob", [1, 2, "a"]]

I've tried using SHLEX, but it seems to be too simple. It does not split parenthesis into different arrays, and does not split commas correctly.

NOW, I know I could maybe do this with some for loops, and some trickery, but I would like to save space, time, and speed for this. I would appreciate some help, thanks!

P.S. It should be able to split multiline, so if I have functions, for example.

"   desu(1, 2){    \n
        asdf       \n
    }               "

Should split as

["desu", [1, 2], ["asdf"]]

Splitting curley braces into different lists as well.

This would save a lot of time! Thanks!

Upvotes: 0

Views: 136

Answers (1)

Hans Then
Hans Then

Reputation: 11322

Have you looked at pyparsing? This is a generic python parsing module that may be able to help you out. Pyparsing can be used for recursive descent parsing, such as programming languages. For languages that do not follow one of the already implemented examples, the pyparsing module has rather steep learning curve. However, for your language, I think you can probably reuse one of the examples on their website.

Upvotes: 1

Related Questions