Current focus is on error reporting and tracking in OILexer.
Code appears to notice when an error occurs and the error cascades up the line, but there needs to be a concentrated effort to provide this context to the end-user on what exactly occurred, and a mechanism for the system to recover from these errors.
Context on every possible outcome is available within the core of OILexer, it just needs exported as a sequence of actions that handle either skipping symbols, or simply breaking if we're already in good standing at the point of an error.
Possible means: trace all edges from all invalid exit states and create distinct subsets that yield the same edges. Use these edges to generate a full set of follow information and seek forward until one of those is hit.
Reduce the look-ahead stack once a valid edge is hit, EoF is hit or the tokenizer encounters an error of its own. This has the chance to yield the entire remainder of the stream, but all parent rules hitting eof will terminate and the error can be reported. Follow edge context will be used to check the information coming from the Lexer, a special 'error recovery' state will need entered that represents all possible tokens from the GetValidSymbols on the Parser.
Code appears to notice when an error occurs and the error cascades up the line, but there needs to be a concentrated effort to provide this context to the end-user on what exactly occurred, and a mechanism for the system to recover from these errors.
Context on every possible outcome is available within the core of OILexer, it just needs exported as a sequence of actions that handle either skipping symbols, or simply breaking if we're already in good standing at the point of an error.
Possible means: trace all edges from all invalid exit states and create distinct subsets that yield the same edges. Use these edges to generate a full set of follow information and seek forward until one of those is hit.
Reduce the look-ahead stack once a valid edge is hit, EoF is hit or the tokenizer encounters an error of its own. This has the chance to yield the entire remainder of the stream, but all parent rules hitting eof will terminate and the error can be reported. Follow edge context will be used to check the information coming from the Lexer, a special 'error recovery' state will need entered that represents all possible tokens from the GetValidSymbols on the Parser.