Cached semantic output

Composing application values

These helpers belong to the bundled Extraction expansion, which loads automatically on first use through Comprehend or Syntax. Extraction::syntax() and C::syntax()->extend(new Extraction()) remain available for explicit local registration.

Follow Parse application values for a complete domain-object example with optional settings, alternatives and nested lists.

map(callable(mixed): mixed) transforms the current semantic value. Unlike mapText() and mapValue(), consecutive map() calls compose in order. Every stage is lazy and caches its value or thrown error for that recognition. Calling mapText() or mapValue() replaces the projection and clears earlier transforms. Low-level Parser objects also expose map().

Extraction also offers mapWith(next, combine) for binary value pipelines and foldLeft(step, combine, min: 0, max: null) for left-associative accumulation. They compose the existing sequence/repetition runtime with lazy cached values, without evaluating callbacks to select syntax. See pipelines and folds and the runnable example.

$syntax->collect(...$operands) recognizes a sequence and returns an ordered list of its direct operand values. Automatic trivia is excluded; nested collects remain nested. At least one operand is required. $syntax->valueChoice(...$alternatives) recognizes an ordinary choice but forwards the selected alternative's value, including null. First/longest/shortest policies and diagnostics remain unchanged; discarded alternatives' mappings are not evaluated to select a match.

use Vanderlee\Comprehend\Extensions\Extraction\Extraction;


$amount = Extraction::syntax()->integer()->mapText('intval');
$currency = Extraction::syntax()->valueChoice('USD', 'EUR');
$money = Extraction::syntax()->collect($currency, $amount)
    ->map(static fn(array $parts): object => (object) [
        'currency' => $parts[0],
        'amount' => $parts[1],
    ])
    ->spacing(Extraction::syntax()->text(' ')->many());

$result = $money->parse('USD 12');
$result->value()->amount; // 12 (int)
$result->rawText();       // 'USD 12'

Both factories accept facades, Parser objects, or literal strings. Syntax vocabularies expose them too. Fluent $a->collect($b) and $a->valueChoice($b) include $a as the first operand and preserve nesting. Ordinary sequence(), then() and choice() retain their raw-text defaults. Low-level constructors are new Collect(Parser ...$operands) and new ValueChoice(Parser ...$alternatives) in Extensions\Extraction\Parser.

These operations preserve the complete CST, source spans, named collections, semantic validators and explicit commit effects. Value inspection never commits. Mapping a collected sequence evaluates operand values in recognition order. An outer mapText() or mapValue() can replace collection/choice projection without evaluating its operand mappings. Run php docs/examples/value-composition.php.

These names are automatically discovered, not reserved core built-ins. Explicitly installing Extraction follows the ordinary Syntax replacement and isolation rules.

Repeated and optional values

Use $item->manyValues() for zero or more item values, $item->someValues() for one or more, or $item->repeatValues(min: 2, max: 4) for explicit bounds. The default projection is an ordered list, including nulls and nested item values; automatic trivia is excluded. An empty accepted repetition yields [].

$item->optionalValue(default: 'missing') returns the accepted item's value when present, and the supplied default when absent. The default is null if omitted. Presence is determined by recognition: accepted null, false, zero and empty-string values do not trigger the default. Defaults are literal values, including callable and object values: callables are never invoked, and objects are not cloned or constructed afresh per parse. Use a pure map() if a fresh application object is needed.

use Vanderlee\Comprehend\Extensions\Extraction\Extraction;

$digit = Extraction::syntax()->range('0', '9')->mapText('intval');
$digit->manyValues()->parse('123')->value(); // [1, 2, 3]
$digit->manyValues()->parse('')->value();    // []
$digit->someValues()->parse('')->success;   // false
$digit->optionalValue(default: 9)->parse('')->value(); // 9
$digit->optionalValue(default: 9)->parse('0')->value(); // 0

Static and Syntax forms accept facades, Parser objects or literal strings: C::manyValues($item), C::someValues($item), C::repeatValues($item, min: 2, max: 4) and C::optionalValue($item, default: 'missing'). These names are discovered Extraction factories, not reserved core built-ins. Explicit Syntax registrations follow the ordinary replacement and isolation rules. Existing many(), some(), repeat(), maybe() and optional() keep their raw-text defaults. Low-level constructors in Extensions\Extraction\Parser are new ValueRepeat(Parser $item, int $min = 0, ?int $max = null) and new OptionalValue(Parser $item, mixed $default = null).

All use the existing greedy repetition runtime: a failed extension rolls back its syntax, spacing, validators and effects. Prefix parsing may leave an incomplete item unconsumed; full-input parsing rejects leftover input. These operations do not recover malformed optional fields. Required failures retain ordinary repetition diagnostics; termination probes do not pollute expectations. Items must consume input, including optional items. Known nullable items are rejected during freezing, with runtime progress checks too; a zero maximum skips the item entirely.

Values and mapping errors remain lazy and cached per recognition. map() composes with the default list/item projection; mapText() and mapValue() replace it. Accepted item validation, collections and explicit commit effects are preserved. CST nodes, trivia and byte spans remain intact. Run php docs/examples/value-repetition.php for a complete application object example.

Static value types

The facade, frozen grammar and parse result carry a PHPDoc value type: Comprehend<T>, FrozenGrammar<T> and ParseResult<T>. Mapping signatures are provided through ordinary PHPDoc, including generic mapping signatures and generated factory declarations. No executable PHPStan extensions or configuration includes are needed. Remove the former phpstan-extension.neon and extensions/Extraction/phpstan.neon includes from consumer configurations. PHPStan is not a runtime dependency. Bundled Extraction signatures are visible to analysis even on an unextended vocabulary, and runtime factories load automatically on first use. Explicit installation is optional.

use Vanderlee\Comprehend\Extensions\Extraction\Extraction;

$number = Extraction::syntax()->integer()->mapText('intval'); // Comprehend<int>
$doubled = $number->map(static fn(int $n): int => $n * 2);
$list = $doubled->manyValues();           // Comprehend<list<int>>
$optional = $number->optionalValue(default: false); // Comprehend<bool|int>
$result = $list->freeze()->parse('123');  // ParseResult<list<int>>
$values = $result->value();              // list<int>|null

map() checks its callback against the current value type and infers the new type. mapText() accepts string callbacks; mapValue() accepts ParseResult callbacks for source/result inspection, not access to an earlier value projection. Fluent someValues() infers a nonempty list; fluent manyValues() and repeatValues() preserve their item type. Fluent optional values include the default's inferred type (which may widen literals), or null when omitted.

collect() returns list<mixed>, record() returns array<array-key, mixed>, and valueChoice() returns mixed. Extraction repetition factories return list<mixed> and optional factories return mixed. Overloaded separatedBy() returns list<mixed> and ignoreThen() returns mixed. These conservative contracts replace the former argument-dependent tuple, shape and union inference. Names, spacing, delimiters and thenIgnore() preserve the selected value type. freeze(), complete and prefix parsing, value() and output() retain it. The application tutorial's jobParser() declares Comprehend<Job> and is checked alongside the inference fixtures.

value() and output() include null for failure, even after inspecting the boolean success property. Narrow the value itself when needed; successful parsers may also produce null. Static types do not guarantee semantic validity or prevent mapping exceptions. Low-level parser inputs conservatively fall back to mixed; arbitrary runtime syntax registrations are not inferred. FrozenGrammar::from() remains mixed; use a facade's freeze() to retain its value type.

The repository uses PHPStan level 8. Levels 9 and 10 reject passing the mixed collection positions directly to the tutorial's typed domain constructor; callers using those levels should validate their collected values at that boundary.

Run make phpstan to check the implementation, positive type assertions and rejection of incompatible callbacks. Without Make, run the analysis command in contributing followed by php tools/check-value-types.php.

Mappings and named collections

mapValue(callable(ParseResult): mixed) adds a lazy typed mapping. value() caches scalars, null, arrays, objects and thrown errors for that recognition. Without a mapping it returns raw text. Transparent runtime wrappers preserve their accepted child's mapping unless explicitly mapped. Failed results return null; abandoned alternatives do not evaluate their mappings automatically.

Value-selecting sequences provide a different default: thenIgnore() returns the first operand's value, ignoreThen() the second, and between() the body's. They retain the full outer raw text, syntax and lifecycle. See delimiters and value selection.

Facade mapText(callable(string): mixed) is the text-only convenience: its callback receives the original matched string instead of ParseResult. It uses the same lazy value cache, supports typed return values, and never changes raw source text or commits effects. Both methods set the same mapper; a later call replaces the previous mapping rather than transforming its output.

Named collections remain useful supported projections: setResult(name, value), concatResult(name, value) and pushResult(name, value). A callable value maps raw text. results() returns the named map; result() accesses the default collection, or result(name, default) accesses a named value. A transform is evaluated once per recognition even when both local semantic validation and a parent collection read need it. Errors are retained and recursive evaluation is rejected. A stored null is distinct from a missing key when using a default. Collection operations do not affect raw text, CST or AST.

Concatenation accepts scalars, null, resources and objects implementing Stringable, using their PHP string representation. Arrays and other objects raise UnexpectedValueException when the collection is read; the error is cached. When concatenating onto a pushed collection, only its last item is concatenated. setResult() and pushResult() continue to retain arbitrary values.

Definition processors receive the local result collection and produce a named value. They replace local collection output, retaining child syntax, validation and effects. Processor results/errors are cached per recognition; they do not depend on mutable parent output. Definition validators receive ParseResult under the selected semantic lifecycle.

Later grammar mutations do not change accepted mappings, collection operations, names, validators or effects. Builders themselves remain mutable until Phase 3; each separate recognition has its own output caches.