Package eu.bandm.tools.lexic

Interface LookaheadTokenSource<D,T,L>

Type Parameters:
D - the type of source document identifiers
T - the type of token types
L - the type of lookahead data
All Superinterfaces:
Supplier<Token<D,T>>
All Known Subinterfaces:
TokenSource<D,T>
All Known Implementing Classes:
LookaheadTokenFilter, LookaheadTokenMultiplexer, LookaheadTokenProcessor, Screener, TokenFilter, TokenProcessor
public interface LookaheadTokenSource<D,T,L> extends Supplier<Token<D,T>>
A supplier of tokens with internal lookahead buffer.

Lookahead data can be transferred from the internal buffer of one instance of this interface to another. This is useful for transfer of control if both share the same input source.

  • Method Details

    • relinquishLookahead

      L relinquishLookahead()
      Clear the internal lookahead buffer and return its contents.
      Returns:
      the former contents of the internal lookahead buffer

    • takeOverLookahead

      void takeOverLookahead(L lookaheadData)
      Fill the internal lookahead buffer with data.

      The buffer must be empty before this method is invoked. After return, the given data will be consumed from the buffer before the usual input is considered.

      Parameters:
      lookaheadData - the lookahead data to fill in
      Throws:
      IllegalStateException - if the internal lookahead buffer is not empty

    • transferLookahead

      static <A> void transferLookahead(LookaheadTokenSource<?,?,? extends A> first, LookaheadTokenSource<?,?,? super A> second)

    • forgetLookahead

      default TokenSource<D,T> forgetLookahead()

    • removeTypes

      default LookaheadTokenSource<D,T,L> removeTypes(T... types)
      Returns a secondary token source that forwards all tokens from this token source that have none of the given types.
      Parameters:
      types - the types of tokens to remove
      Returns:
      a token source that forwards all tokens from this token source that have none of the given types

    • removeTypes

      default LookaheadTokenSource<D,T,L> removeTypes(Set<T> types)
      Returns a secondary token source that forwards all tokens from this token source that have none of the given types.
      Parameters:
      types - the types of tokens to remove
      Returns:
      a token source that forwards all tokens from this token source that have none of the given types