Package eu.bandm.tools.lexic

Class LookaheadTokenProcessor<D,T,L>

java.lang.Object
eu.bandm.tools.lexic.LookaheadTokenProcessor<D,T,L>
Type Parameters:
D - the type of source document identifiers
T - the type of token types
L - the type of lookahead data
All Implemented Interfaces:
LookaheadTokenSource<D,T,L>, Closeable, AutoCloseable, Supplier<Token<D,T>>
Direct Known Subclasses:
LookaheadTokenFilter, Screener, TokenProcessor
public abstract class LookaheadTokenProcessor<D,T,L> extends Object implements LookaheadTokenSource<D,T,L>
Abstract base class for secondary token sources that feed on other token sources.

  • Field Details

  • Constructor Details

    • LookaheadTokenProcessor

      protected LookaheadTokenProcessor(LookaheadTokenSource<D,T,L> input)
      Creates a new instance.
      Parameters:
      input - the input token source

  • Method Details

    • relinquishLookahead

      public L relinquishLookahead()
      Description copied from interface: LookaheadTokenSource
      Clear the internal lookahead buffer and return its contents.
      Specified by:
      relinquishLookahead in interface LookaheadTokenSource<D,T,L>
      Returns:
      the former contents of the internal lookahead buffer

    • takeOverLookahead

      public void takeOverLookahead(L lookaheadData)
      Description copied from interface: LookaheadTokenSource
      Fill the internal lookahead buffer with data.

      The buffer must be empty before this method is invoked. After return, the given data will be consumed from the buffer before the usual input is considered.

      Specified by:
      takeOverLookahead in interface LookaheadTokenSource<D,T,L>
      Parameters:
      lookaheadData - the lookahead data to fill in

    • close

      public void close()
      Closes this source and releases any resources associated with it.

      The default implementation does nothing.

      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Specified by:
      close in interface LookaheadTokenSource<D,T,L>