This page is no longer maintained — Please continue to the home page at

Re: Parser Combinators

No replies
Florian Hars
Joined: 2008-12-18,
User offline. Last seen 42 years 45 weeks ago.

Daniel Swe schrieb:
> But how would you proceed? Which parts do I need to change for it to
> accept an underscore as a letter?

You are using StandardTokenParsers which is just:

class StandardTokenParsers extends StdTokenParsers {
type Tokens = StdTokens
val lexical = new StdLexical

You'd have to replace this with a class that uses something else as
lexical that has a different token parser from

class StdLexical extends Lexical with StdTokens {
// see `token' in `Scanners'
def token: Parser[Token] =
( letter ~ rep( letter | digit ) ^^ { case first ~ rest => processIdent(first :: rest mkString "") }

(letter | '_') ~ rep (letter | '_' | digit) ^^ {...})

might be the minimal change. But be careful that with this definition, both
aʹ and aʼ are valid (and different) identifiers, while a' isn't.
And of course,
scala.util.parsing.combinator.lexical.Lexical parses Chars which means
you are limited to the BMP. So yes, japanese works, cuneiform or ugaritic wont.

I can't reproduce your \t problem, so no help there.

- Florian.

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland