Reading and skipping runs¶
Read scanners consume a contiguous run in one terminal invocation and return its
original text. The bundled Optimizer extension supplies all read/skip factories
and loads automatically on first use. They use the same checked runtime as other
terminals and remain available with FreezeFlags::NONE; that flag disables graph
optimization, not explicitly authored scanners.
Direct parser classes live in Extensions\Optimizer\Parser: ReadRun (the
abstract base), ReadWhile, ReadUntil, ReadWhileSet, ReadWhileRange, and
ReadRest. Their former Parser\Terminal names have no compatibility aliases.
Scanner CST kinds use the extension class names. Factory signatures and static,
scoped Syntax, and fluent calls retain their behavior. These factory names are
discoverable rather than reserved core names and can be overridden in a local
Syntax vocabulary.
use Vanderlee\Comprehend\Comprehend as C;
C::readWhile(static fn(string $character): bool => ctype_alpha($character), min: 1);
C::readWhileSet(" \t\r\n");
C::readWhileRange('0', '9', min: 1, max: 8);
C::readUntil(';');
C::readUntil('*/');
C::readUntil(static fn(string $character): bool => $character === ';');
C::readRest();
min defaults to zero and max to null (unlimited). Bounds count Unicode code
points for UTF-8 sources and octets for byte sources, while offsets and spans
remain byte-based. Fewer than min accepted units causes failure without
consumption, with diagnostics at the observed stopping position. Reaching max
succeeds without probing the next unit. Negative bounds or max < min are
rejected at construction.
readUntil() leaves its delimiter unconsumed. It allows an empty prefix and
succeeds at EOF if the delimiter is absent, provided min is satisfied. A string
always denotes a nonempty literal, even if it names a PHP function. Pass a closure
or other non-string callable for predicates. A delimiter beginning exactly at a
finite maximum need not be inspected. These bounds count content, not delimiter
units. To require and consume the delimiter while retaining only the content:
use Vanderlee\Comprehend\Extensions\Extraction\Extraction;
$content = Extraction::syntax()->readUntil(';')->thenIgnore(';');
$content->parse('hello;')->value(); // 'hello'
Predicates must be pure and return bool. They receive one raw source unit, including whitespace and newlines, without case folding. Set membership, ranges, and literal delimiters inherit the scoped case policy and support local overrides. Ranges are inclusive and may have one null endpoint; the other endpoint must be one source unit. Bounds must remain ordered after case folding, and folded range units must remain single code points. Source-dependent validation happens when recognition starts, so arbitrary byte sets/delimiters remain usable in byte mode.
skipWhile(predicate, min: 0, max: null),
skipUntil(delimiter, min: 0, max: null), and skipRest() recognize the same
input as their read counterparts but return null. Skipping only discards the
semantic value: source spans, CST text, named AST content, diagnostics, and
explicit commit effects remain available. Like other terminal factories, these
methods work through C::syntax() and as fluent terminal chaining.
Each scanner produces a single CST terminal node for the entire run. It does not
insert spacing between units. Use a scanner as the grammar's spacing parser to
scan intervening trivia, or compose it with other parsers normally. Nullable
scanners must not be placed inside unbounded repetition; use min: 1 when
progress is required. Work limits account for scanning and delimiter lookahead.
Case-sensitive sets, ranges and literal delimiters use cached regex patterns for
in-memory input. The generated patterns preserve empty matches, unit-count bounds,
byte positions and the single-terminal CST shape. Predicates still run as PHP
callbacks; readRest() uses direct source slicing rather than a regex.
The source-unit loop remains the fallback for case-insensitive matching, file or custom sources, finite work budgets, finite maxima above PCRE's repeat limit, generated atoms larger than 8 KiB, and regex engine failures. This keeps scoped case folding, cooperative work-limit positions and source access behavior intact. Skip methods share the same recognition paths as their read counterparts.
See the runnable example.