Package eu.bandm.tools.lexic

Class TokenFilter<D,T>

java.lang.Object

eu.bandm.tools.lexic.LookaheadTokenProcessor<D,T,Void>

eu.bandm.tools.lexic.LookaheadTokenFilter<D,T,Void>

eu.bandm.tools.lexic.TokenFilter<D,T>

Type Parameters:

D - the type of source document identifiers

T - the type of token types

All Implemented Interfaces:

LookaheadTokenSource<D,T,Void>, TokenSource<D,T>, Supplier<Token<D,T>>

public class TokenFilter<D,T>
extends LookaheadTokenFilter<D,T,Void>
implements TokenSource<D,T>

Abstract base class for token processors that filter out certain tokens. Tokens from the given input token source are either forwarded or discarded, depending on a variety of criteria.

Field Summary

Fields inherited from class eu.bandm.tools.lexic.LookaheadTokenProcessor

input

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	TokenFilter(TokenSource<D,T> input, Predicate<? super Token<D,T>> acceptable)	Creates a new instance.

Method Summary

Modifier and Type	Method	Description
static <D, T> TokenFilter<D,T>	discard(TokenSource<D,T> input, Predicate<? super Token<D,T>> pred)
static <D, T> TokenFilter<D,T>	removeTypes(TokenSource<D,T> input, Set<? extends T> types)

Methods inherited from class eu.bandm.tools.lexic.LookaheadTokenFilter

discard, forward, get, removeTypes, removeTypes

Methods inherited from class eu.bandm.tools.lexic.LookaheadTokenProcessor

relinquishLookahead, takeOverLookahead

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface eu.bandm.tools.lexic.LookaheadTokenSource

forgetLookahead

Methods inherited from interface java.util.function.Supplier

get

Methods inherited from interface eu.bandm.tools.lexic.TokenSource

relinquishLookahead, removeTypes, removeTypes, takeOverLookahead, with

Constructor Details

TokenFilter

protected TokenFilter(TokenSource<D,T> input,
 Predicate<? super Token<D,T>> acceptable)

Creates a new instance.

Parameters:

input - the input token source

Method Details

discard

public static <D,
T> TokenFilter<D,T> discard(TokenSource<D,T> input,
 Predicate<? super Token<D,T>> pred)

removeTypes

public static <D,
T> TokenFilter<D,T> removeTypes(TokenSource<D,T> input,
 Set<? extends T> types)