A two-pass parser for a language where "sin x", "2x", and "3 add 4" are all legal
How Equana parses bracket-free calls, implicit multiplication, and user-defined infix operators without backtracking — a registry scan followed by a plain Pratt parser.
Equana lets you write sin x instead of sin(x), 2x instead of 2 * x, and 3 add 4 instead of add(3, 4) — and it lets you turn any two-argument function of your own into an infix operator with a precedence you choose. Each of those features is nice on its own. Together they look like a parser's nightmare, because they make the meaning of a bare sequence of tokens depend on what the identifiers in it are.
Take a foo b. Is it an infix call foo(a, b)? Or a * foo(b) — implicit multiplication of a by the bracket-free call foo b? Or a * foo * b, three values multiplied? Every one of those is a valid Equana expression for some meaning of foo. You cannot decide which by looking at the punctuation, because there isn't any. The token stream is genuinely ambiguous until you know whether foo is a function, an infix operator, or a plain value.
This is the classic trap that makes real-world grammars miserable. The C grammar is famously not context-free because a * b means multiplication or a pointer declaration depending on whether a is a type name — the "lexer hack" wires the symbol table back into the lexer to cope. Once you let context leak into parsing, you tend to end up with backtracking, a parser that re-reads input trying interpretations until one sticks, and error messages that make no sense because the parser doesn't know which interpretation you meant.
Equana avoids all of that. The trick is to make the context explicit and static before expression parsing begins, so that the expression parser itself stays a completely ordinary, context-free, backtracking-free Pratt parser. Two passes. That's the whole idea.
Pass 1: the registry scan
The first pass is deliberately not parsing. It's a linear O(n) walk over the raw token array that does pattern-matching only — no recursion, no tree, no grammar. Its entire job is to build one small object:
interface InfixRegistry {
// User-defined infix function name → binding power (precedence).
infixOps: ReadonlyMap<string, number>
// All declared function names (to disambiguate implicit-multiply vs. call).
functionNames: ReadonlySet<string>
}
The scan looks for exactly two patterns as it slides down the tokens:
- Function declarations. Every
functionkeyword is followed by a name — whether it'sfunction name(...),function r = name(...), orfunction (r1, r2) = name(...). The scanner knows those three shapes and extracts the name, adding it tofunctionNames.exportandglobalmodifiers are skipped over. @infix(n)decorators. The pattern@ infix ( <int> )before a function declaration records that function's name with precedencenininfixOps.
The set of built-in function names and the default infix operators are seeded in first, so from the parser's perspective there's no difference between sin (built-in) and a function you defined three lines up. And because user @infix(N) declarations are applied over the defaults, precedence is always something the user can control.
The important property is what the scan does not do: it never parses a function body, never builds an expression, never recurses. It's pattern matching over a flat array. That's what keeps it cheap enough to run on every cell before every real parse, and it's what lets a function be used before its declaration is reached — the name is already in the set by the time the second pass looks.
Pass 2: a plain Pratt parser with a lookup
The second pass is a textbook Pratt (top-down operator-precedence) parser. The core of Pratt parsing is that each infix operator has a binding power — a number saying how tightly it grabs its operands — and the parser recurses while the next operator binds tighter than the current context. Equana's binding-power table is unremarkable and lives in one place:
ARROW < OR < AND < EQUALITY < COMPARISON < RANGE
< ADDITIVE < MULTIPLICATIVE < POWER < UNARY < POSTFIX < CALL
So + and - sit at ADDITIVE, * / % at MULTIPLICATIVE (tighter, as expected), ^ at POWER and right-associative, and CALL — function application and indexing — binds tightest of all. 1 + 2 * 3 parses as 1 + (2 * 3) for the same reason it does in every other language: multiplication's binding power is higher.
What makes this parser handle Equana's optional syntax is that the two ambiguity-resolving rules are not special grammar productions — they're a getInfixBP lookup consulting the registry from pass 1. When the parser is sitting on a value and sees an identifier foo next, it asks three questions in order:
- Is
fooa registered infix operator? If so, it's an infix call atfoo's declared binding power.3 add 4→add(3, 4);3 cross 4→cross(3, 4)at whatever@infix(n)said. - Is
fooa known function name? Then function-call priority applies: a function name followed by an expression is a call.sin x→sin(x), neversin * x. This is what makessin cos 0.0parse assin(cos(0.0))— bracket-free calls chain right-to-left. - Otherwise, a value followed by a value is implicit multiplication.
2x→mul(2, x),2 pi→mul(2, pi).
Every one of those decisions is a map lookup, resolved at the moment the token is read. There is no point at which the parser guesses, produces a tree, discovers it was wrong, and rewinds. The grammar of expressions stays context-free; all the context has been pre-baked into one flat name → precedence table and one name set. That's the payoff of splitting the work: the messy, context-sensitive part is a dumb linear scan, and the recursive part is boringly regular.
Here's the same call written all the ways the parser accepts, all producing the same AST and dispatching identically:
@infix(10)
function r = cross(a, b) { r = a * b }
cross(3, 4) # ordinary call
3.cross(4) # method syntax — cross(3, 4)
3 cross 4 # infix — cross(3, 4)
The ambiguities that forced compromises
Making this work without backtracking meant giving up some things I'd have liked to keep. These are the real edges, stated as design decisions rather than regrets.
Parentheses force a call. foo(x) is unconditionally a function call, even in a context where foo x would have been implicit multiplication. That's the escape hatch — when the heuristics would read your line the wrong way, parentheses let you say exactly what you mean. The cost is that (a)(b) and a b are not interchangeable, which is a rule you have to learn.
end and in are reserved outright. in aliases the range/annotation colon in the natural-language layer and drives for x in xs; end closes blocks. Letting them also be identifiers would reintroduce exactly the context-sensitivity the two-pass design exists to avoid, so they're keywords, and you can't name a variable end.
User precedence must be explicit. @infix(n) makes you write the number. I considered inferring precedence — grouping user operators into tiers, or defaulting them all to one level — and every option produced the operator soup where nobody can read a foo b bar c without a manual. Forcing the number is friction with a purpose: the precedence of your operator is information only you have, so you state it.
Implicit multiplication and bracket-free calls are the danger zone. 2 sin x is 2 * sin(x) only if you know both rules and their priority. I keep this feature because the audience is scientific — code is dense with 2πr-shaped expressions that people already read this way on paper — but in a general-purpose language I wouldn't ship it. The two-pass parse makes the feature unambiguous to the machine; it can't make it obvious to a reader who doesn't know the rules.
Why two passes instead of one clever one
You could imagine folding this into a single pass with enough lookahead and state. I didn't, for a reason that matters more than elegance: separation makes each pass simple enough to be obviously correct. Pass 1 has one job and no recursion; you can read it top to bottom and convince yourself it collects the right names. Pass 2 is a standard algorithm out of any compilers text, with the Equana-specific behavior isolated to three lines of getInfixBP. Neither pass has to reason about the other's edge cases. When a parse goes wrong, it's wrong in a small, local way, and the error points at a real token instead of at the wreckage of an abandoned interpretation.
That's the same instinct that runs through the rest of the interpreter — favor a design where each piece is small and legible over one that's maximally clever — and it's why a language with genuinely ambiguous surface syntax still parses in a single deterministic sweep with no backtracking.
You can try every spelling in this post at equana.dev (alpha, free, sign-in for the full notebook). The philosophy behind the optional syntax is in designing a language that reads like English, and once the parser has produced add(a, b), how the language chooses which add to run is the subject of multiple dispatch in a dynamic language.