Package eu.bandm.tools.d2d2.parser2

Class TLex<D,T>

java.lang.Object
eu.bandm.tools.d2d2.parser2.TLex<D,T>
All Implemented Interfaces:
Supplier<Parser.Token<D,T>>
public class TLex<D,T> extends Object implements Supplier<Parser.Token<D,T>>
    Item
       continuation  Item
       action (TLex context)
    | ConsumeOrShift
       c     char
       cset  CharSet
       actionAfterTest(TLex context)
    | | Consume
    | | Shift
    | ChoiceOrState
       alternatives SEQ ConsumeOrShift
       other        OPT Otherwise
       actionAfterTest(TLex context)
    | | Choise
    | | State
    | Other
    | CloseState
    | Append
        text        OPT string
    | Deliver
        reduce      OPT Function
        t           TokenType
    | Text2Token
        fun         BiFunction<String,Location,Tokentype>
    | Text2cont
        fun         BiFunction<String,Location,Tokentype>
    | Error
        text        String

  • Field Details

  • Constructor Details

  • Method Details

    • setInput

      public void setInput(D documentid, Reader reader)

    • get

      public Parser.Token<D,T> get()
      Operation: As long as no token as yet been recognized, the item in the field "TLex.continuation" is executed, i.e. its "action()" method is called. Every such action copies its own ".continuation" into that field (Cannto be factored out, because State and Choice behave differentyl: the continuation is defined by the selected child element.) If this is ==0, then the continuation is pulled from the stateStack.
      Specified by:
      get in interface Supplier<D>

    • consume

      public static <DD, TT> TLex.Consume<DD,TT> consume(char c, @Opt @Opt TLex.Item<DD,TT> continuation)

    • consume

      public static <DD, TT> TLex.Consume<DD,TT> consume(char c, TT type)

    • consume

      public static <DD, TT> TLex.Consume<DD,TT> consume(TunableParser.CharSet cs, @Opt @Opt TLex.Item<DD,TT> continuation)

    • consume

      public static <DD, TT> TLex.Consume<DD,TT> consume(@Opt @Opt TLex.Item<DD,TT> continuation)

    • consume

      public static <DD, TT> TLex.Consume<DD,TT> consume()

    • consume

      public static <DD, TT> TLex.Consume<DD,TT> consume(char c)

    • consume

      public static <DD, TT> TLex.Consume<DD,TT> consume(TunableParser.CharSet cs)

    • shift

      public static <DD, TT> TLex.Shift<DD,TT> shift(char c, @Opt @Opt TLex.Item<DD,TT> continuation)

    • shift

      public static <DD, TT> TLex.Shift<DD,TT> shift(@Opt @Opt TLex.Item<DD,TT> continuation)

    • shift

      public static <DD, TT> TLex.Shift<DD,TT> shift()

    • shift

      public static <DD, TT> TLex.Shift<DD,TT> shift(TunableParser.CharSet cs, @Opt @Opt TLex.Item<DD,TT> continuation)

    • shift

      public static <DD, TT> TLex.Shift<DD,TT> shift(char c)

    • shift

      public static <DD, TT> TLex.Shift<DD,TT> shift(TunableParser.CharSet cs)

    • choice

      @SafeVarargs public static <DD, TT> TLex.Choice<DD,TT> choice(TLex.Item<DD,TT>... items)

    • state

      @SafeVarargs public static <DD, TT> TLex.State<DD,TT> state(TLex.Item<DD,TT>... items)

    • other

      public static <DD, TT> TLex.Other<DD,TT> other(TLex.Item<DD,TT> continuation)

    • other

      public static <DD, TT> TLex.Other<DD,TT> other()

    • closeState

      public static <DD, TT> TLex.CloseState<DD,TT> closeState()

    • closeState

      public static <DD, TT> TLex.CloseState<DD,TT> closeState(TLex.Item<DD,TT> continuation)

    • append

      public static <DD, TT> TLex.Append<DD,TT> append(@Opt @Opt String s, @Opt @Opt TLex.Item<DD,TT> continuation)

    • append

      public static <DD, TT> TLex.Append<DD,TT> append(@Opt @Opt String s)

    • append

      public static <DD, TT> TLex.Append<DD,TT> append(@Opt @Opt TLex.Item<DD,TT> continuation)

    • append

      public static <DD, TT> TLex.Append<DD,TT> append()

    • deliver

      public static <DD, TT> TLex.Deliver<DD,TT> deliver(TT t, @Opt @Opt TLex.Item<DD,TT> continuation)

    • deliver

      public static <DD, TT> TLex.Deliver<DD,TT> deliver(TT t)

    • text2token

      public static <DD, TT> TLex.Text2token<DD,TT> text2token(BiFunction<String,Location<DD>,Parser.Token<DD,TT>> fun)
      Applies a user function to the accumulated text.

    • text2token

      public static <DD, TT> TLex.Text2token<DD,TT> text2token(BiFunction<String,Location<DD>,Parser.Token<DD,TT>> fun, @Opt @Opt TLex.Item<DD,TT> continuation)

    • text2cont

      public static <DD, TT> TLex.Text2cont<DD,TT> text2cont(BiFunction<String,Location<DD>,TLex.Item<DD,TT>> fun)
      Applies a user function to the accumulated text.

    • error

      public static <DD, TT> TLex.Error<DD,TT> error(String t)

    • verbatimLike

      public static <DD, TT> TLex.Consume<DD,TT> verbatimLike(TT tt, String pre, String post)
      Accepts token contents delimited by two string, like the LaTeX "verbatim" command. ATTENTION: Repetitions of characters in the limiting strings are not supported. The automaton built by verbatimLike("ab","yz") has the following structure: 
        CL() stands for closeState() in:
 
  consume('a',state(consume('b', state(consume('y',state(consume('z',CL(CL(CL(deliver(tt))))),
                                                         other(append('y',shift(CL)))
                                                        )),
                                       other(shift())
                                       ),        
                    other(append('a',CL()))
                   )))
      The resulting automaton shall be included in a choice() or state() disjunction.
      ATTENTION NOT YET TESTED, never used.

    • verbatimEnd

      public static <DD, TT> TLex.State<DD,TT> verbatimEnd(TT tt, String post, @Opt @Opt TLex.Item<DD,TT> continuation)
      Accepts a verbatim input character sequence ended by a particular string constant. ATTENTION: Repetitions of characters in the limiting string are not supported. The automaton built by verbatimLike("ab","yz") has the following structure: 
        CL() stands for closeState() in:
 
  state(consume('y',state(consume('z',CL(CL(deliver(tt, cont)))),
                          other(append('y',shift(CL)))
                         )),
        other(shift())
        )

    • closeStates

      public static <DD, TT> TLex.CloseState<DD,TT> closeStates(@Opt @Opt TLex.Item<DD,TT> cont, int times)

    • tokenIgnorer

      @SafeVarargs public static final <DD, TT> Supplier<Parser.Token<DD,TT>> tokenIgnorer(Supplier<Parser.Token<DD,TT>> lexer, TT... toIgnore)

    • hexNumber

      public static <DD, TT> TLex.Item hexNumber(TT hexnumberTokenType)
      Recognizes and reduces hexadecimal constant of form "0x[d]" and "0X[d]". [d] stands for an arbitrary, finite, non-empty sequence of hexadeimal digits, upper or lower case.

    • comments

      public static <DD, TT> TLex.Item<DD,TT> comments(TT singleLine, TT multiLine, TT singleSlash)
      Accepts and returns a standard C-/Java-style single line comment. PROVIS FIXME "singleSlash" should be replaced by merging the choices one level above!

    • stringconst

      public static <DD, TT> TLex.Item<DD,TT> stringconst(char delim, TT tt)
      Accepts and returns a standard string constant, containing standard escape sequences. FIXME welche escapes in singlequote/doublequote !?!? Parametrisierbar ??