Package eu.bandm.tscore.base

Class RawParser

java.lang.Object
eu.bandm.tscore.base.RawParser
public class RawParser extends Object
Recursive-descent parser for the raw level of tscore. Consumes tokens from an antlr lexer. Generates a TimeScape object. Roughly the accepted syntax can be described by the following mere symbolic rules (for details see the user documentation): 
  timeScape    ::= conform? timeless* voxParams* part* EOF
  conform      ::= "CONFORM" qident    FIXME CONFORM RAUS   USE "conforma = eu.bandm.music.applciationx.X" instaeD !
  qident       ::= ident ("." ident)*

  timeless     ::= ident "="  qident_or_stringconst  FIXME ANDERS

  voxParams    ::= VOX_and_name conform?, timeless*
  VOX_and_name ::= "VOX"  ident
  part         ::= "PARS"  ident_or_stringconst conform? timeless* voxParams* accolade*
  accolade     ::= "T" timeline NL vox*
  vox          ::= VOX_and_name NL (paramIdent token* NL)

  --- a maximal score can thus look like: ---

  CONFORM xxxx
      a=xxx b=xxx
    VOX A CONFORM xxxx a=xxx b=xxx
    PARS A CONFORM xxx  a=xxx b=xxx
      VOX A CONFORM xxxx a=xxx b=xxx
      T      1   2   3
      VOX A  x   x x x
The implementation is as recursive decent parser, LL(1), explicity inquiring the type of the next Token. This is delivered by a TScoreLexer.

This parser only collects text fragments and assigns them to top time points, timeless parameter settings and parameters of events, which are related to timepoints and voices. No kind of semantical evaluation happens here, but by subsequent interpretation steps.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    protected class 
    RawParser.PosList
    Maps one level of input column numbers either to null, for non-parenthesis entry, or to another instance of this class, for parenthesized sub-expressions, with the opening paranthesis in this column.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected eu.bandm.tools.util.xml.XMLDocumentIdentifier
    docidCache
    Identifier of the last parsed token.
    static final String
    k_CONFORM
     
    static final String
    k_PARS
    Reserved word.
    static final String
    k_T
     
    static final String
    k_VOX
     
    protected Modifiers
    modifiers
    Configuration object controlling this parsing process (and some routines in Util.
    protected MessageReceiver<SimpleMessage<eu.bandm.tools.util.xml.XMLDocumentIdentifier>>
    msg
    Drain of all error and warning messages when parsing.
    protected Token
    nextToken
    Token currently analysed.
    static final Object[]
    noArgs
    Auxiliary: Empty argument list for message call.
    protected Supplier<Token>
    tokenStream
    Input to the parsing process.

  • Constructor Summary

    Constructors
    Constructor
    Description
    RawParser()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected boolean
    accolade(Part current_part)
    Parser function (=test, consume and return) for
    accolade ::= T timeline NL vox* This is the central parsing function which creates time points, events, voices and registers all these in the part under constrction.
    Top time points may use more than one (>1) source line, iff allowed by modifiers.
    protected SortedMap<Integer,List<Token>>
    collectFragments(int firstFragmentColumn, boolean breakAtParentheses)
    Parser function (=test, consume and return) for unparsed, raw token sequence in time lines and parameter lines.
    protected @Opt String
    conform()
    Parser function (=test, consume and return) for
    "conform ::= CONFORM qident"
    protected void
    consume()
    Fill local variable nextToken by pulling from tokenStream.
    protected SortedMap<Integer,String>
    flatten(Map<Integer,List<Token>> tokens, String separator)
    Concatenate source text of time point values which use more than one (>1) lines of input.
    protected SortedMap<Integer,List<Token>>
    foldInto(SortedMap<Integer,RawParser.PosList> target, SortedMap<Integer,List<Token>> data, boolean isTopLevel)
    Folds a sequence of input tokens in the main parameter line of a voice into a sub-tree of RawParser.PosList, controled by opening and closing parentheses.
    protected @Opt eu.bandm.tools.util.xml.XMLDocumentIdentifier
    getDocumentId(Token token)
    Extract the document id from the given token, which is delivered by the tscore lexer TScoreLexer.
    protected eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier>
    getLocation(Token token)
    Get location information in the same way as foreseen for the xantlr/tdom parsing pipeline, which is ...
    protected @Opt String
    ident_or_stringConst()
    Parse (=test, consume and return) TokenType.CHARSTRING or TokenType.IDENT.
    (package private) boolean
    isReserved(String token)
    Whether the token text is a reserved word.
    protected boolean
    isTopTpText(String t)
    Allow everything as top time point which is not contained in the string constants which make up the sub-dividers.
    protected boolean
    LA(TokenType t)
    Tests whether type of nextToken is the given.
    protected boolean
    LA_ident(String s)
    Tests whether the reserved word given is in nextToken.
    protected void
    locError(String m, Object... args)
    Generates an error message with given text and arguments, referring to the next location.
    protected void
    locWarning(String m, Object... args)
    Generates a warning message with given text and arguments, referring to the next location.
    protected void
    makeLowerTps(SortedMap<Integer,Tp> col2tp, String sign, Map<Integer,String> sources, Map<Integer,List<Token>> unpacked)
    Step through T-line source fragments and create TpSubs which divide the interval between already recognized, higher time point levels.
    protected eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier>
    nextLocation()
    Return the location of nextToken.
    TimeScape
    parse(antlr.TokenStream tokenStream, Modifiers modifiers, MessageReceiver<SimpleMessage<eu.bandm.tools.util.xml.XMLDocumentIdentifier>> msg)
    Deprecated.
    TimeScape
    parse(Supplier<Token> tokenStream, Modifiers modifiers, MessageReceiver<SimpleMessage<eu.bandm.tools.util.xml.XMLDocumentIdentifier>> msg)
    Top-level parser for
    timeScape ::= conform? timeless* voxParams* part* EOF.
    protected boolean
    part(TimeScape current_ts)
    Parser function (=test, consume and return) for
    pars ::= PARS ident_or_stringconst conform timeless* voiceparams accolade*
    protected @Opt String
    qident()
    Parse (=test, consume and return) dotted-separated list of idents.
    protected void
    react(Modifiers.Reaction react, eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier> loc, String txt)
    Generates maybe a message with the given text and arguments, referring to the explicitly given location, either as error or as warning, depending on "react".
    protected void
    react(Modifiers.Reaction react, eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier> loc, String txt, Object... args)
    Generates maybe a message with the given text, arguments and location, either as error or as warning, depending on "react".
    protected void
    react(Modifiers.Reaction react, String txt, Object... args)
    Generates maybe a message with the given text and arguments, referring to the next location, either as error or as warning, depending on "react".
    protected SortedMap<Integer,String>
    singleCheck(Map<Integer,List<Token>> tokens)
    Check whether each collected fragment uses only one(1) input line, and return this only text as a (trivial) concatenation result.
    protected String
    stringContent(String s)
    Strip quote symbols and un-quote contained doublequotes.
    protected Timeless
    timelessLine()
    Parses a = b = X and a = b = (c=d e=f 1 2 3)
    protected void
    timelessSequence(List<Timeless> target)
    Parser code (=test, consume, store to target, and return flag) for one timeless item, whch can be a value or an assigment.
    protected @Opt String
    timelessSingleValue()
    Accept all tokens which are not whitespace or parantheses and concatenate their contents.
    (package private) boolean
    voice(Part part, SortedMap<Integer,Tp> col2tp, Set<String> voicesInAccolade)
    Parse function (=test, consume and return) for
    "vox ::= VOX ident token NL (paramIdent token* NL)"
    Is called in the context of parsing an accolade, i.e.
    protected @Opt Vox
    VOX_and_name(Container container, boolean multiple)
    Parse function (=test, consume and return) for
    VOX_and_name ::= VOX ident
    Can be called (a) in context of meta values (in a score or in a part, before the first T-line) or (b) in an accolade after the T-line.
    protected boolean
    voxParams(Container container)
    Parser function (=test, consume and return) for
    voxParams ::= VOX_and_name conform?, timeless* Creates a new voice by calling VOX_and_name(Container,boolean), which stores this either to the timescape or to the part under construction.
    protected boolean
    ws_or_comment()
    Parse (=test, consume and return) whitespace.
    protected boolean
    ws_or_nl_or_comment()
    Parse (=test, consume and return) whitespace.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • k_PARS

      public static final String k_PARS
      Reserved word. Currently there are only four words which cannot be used for voice or parameter names, etc.
      See Also:

    • k_CONFORM

      public static final String k_CONFORM
      See Also:

    • k_T

      public static final String k_T
      See Also:

    • k_VOX

      public static final String k_VOX
      See Also:

    • msg

      protected MessageReceiver<SimpleMessage<eu.bandm.tools.util.xml.XMLDocumentIdentifier>> msg
      Drain of all error and warning messages when parsing.

    • modifiers

      protected Modifiers modifiers
      Configuration object controlling this parsing process (and some routines in Util.

    • docidCache

      protected eu.bandm.tools.util.xml.XMLDocumentIdentifier docidCache
      Identifier of the last parsed token.

    • tokenStream

      protected Supplier<Token> tokenStream
      Input to the parsing process.

    • nextToken

      protected Token nextToken
      Token currently analysed.

    • noArgs

      public static final Object[] noArgs
      Auxiliary: Empty argument list for message call.

  • Constructor Details

    • RawParser

      public RawParser()

  • Method Details

    • parse

      @Deprecated public TimeScape parse(antlr.TokenStream tokenStream, Modifiers modifiers, MessageReceiver<SimpleMessage<eu.bandm.tools.util.xml.XMLDocumentIdentifier>> msg)
      Deprecated.

    • parse

      public TimeScape parse(Supplier<Token> tokenStream, Modifiers modifiers, MessageReceiver<SimpleMessage<eu.bandm.tools.util.xml.XMLDocumentIdentifier>> msg)
      Top-level parser for
      timeScape ::= conform? timeless* voxParams* part* EOF. Returns a time scape. This is a collection of some meta data ("conforms" and "timeless" for the whole score and for voices), followed by the "parts", which hold the real time points and events, each part independently. The meta data from the score level may later be distributed by the particular evaluation code, or ignored, or even explicitly rejected.

    • getDocumentId

      @Opt protected @Opt eu.bandm.tools.util.xml.XMLDocumentIdentifier getDocumentId(Token token)
      Extract the document id from the given token, which is delivered by the tscore lexer TScoreLexer.
      See Also:

    • getLocation

      protected eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier> getLocation(Token token)
      Get location information in the same way as foreseen for the xantlr/tdom parsing pipeline, which is ... 
          generated parser code -> startElement()
                               -> SAXEventGenerator -> SAXEventStream ->  TDOM parse()
                                  locatorInterface <--------         <- getLocation

    • nextLocation

      protected eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier> nextLocation()
      Return the location of nextToken.

    • locError

      protected void locError(String m, Object... args)
      Generates an error message with given text and arguments, referring to the next location.

    • locWarning

      protected void locWarning(String m, Object... args)
      Generates a warning message with given text and arguments, referring to the next location.

    • react

      protected void react(Modifiers.Reaction react, String txt, Object... args)
      Generates maybe a message with the given text and arguments, referring to the next location, either as error or as warning, depending on "react". This allows to choose between "silently allow", "warning", and "error".

    • react

      protected void react(Modifiers.Reaction react, eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier> loc, String txt)
      Generates maybe a message with the given text and arguments, referring to the explicitly given location, either as error or as warning, depending on "react". This allows to choose between "silently allow", "warning", and "error".

    • react

      protected void react(Modifiers.Reaction react, eu.bandm.tools.location.Location<eu.bandm.tools.util.xml.XMLDocumentIdentifier> loc, String txt, Object... args)
      Generates maybe a message with the given text, arguments and location, either as error or as warning, depending on "react". This allows to choose between "silently allow", "warning", and "error".

    • isReserved

      boolean isReserved(String token)
      Whether the token text is a reserved word. Currently these are "PARS", "CONFORM", "VOX, "T".

    • consume

      protected void consume()
      Fill local variable nextToken by pulling from tokenStream.

    • LA

      protected boolean LA(TokenType t)
      Tests whether type of nextToken is the given. Type constants are in TokenType.

    • LA_ident

      protected boolean LA_ident(String s)
      Tests whether the reserved word given is in nextToken.

    • ws_or_nl_or_comment

      protected boolean ws_or_nl_or_comment()
      Parse (=test, consume and return) whitespace.

    • ws_or_comment

      protected boolean ws_or_comment()
      Parse (=test, consume and return) whitespace.

    • ident_or_stringConst

      @Opt protected @Opt String ident_or_stringConst()
      Parse (=test, consume and return) TokenType.CHARSTRING or TokenType.IDENT. Strips quote signs.

    • stringContent

      protected String stringContent(String s)
      Strip quote symbols and un-quote contained doublequotes.

    • qident

      @Opt protected @Opt String qident()
      Parse (=test, consume and return) dotted-separated list of idents. Return null iff none could be parsed (then parser state stays unaltered)

    • conform

      @Opt protected @Opt String conform()
      Parser function (=test, consume and return) for
      "conform ::= CONFORM qident"
      Returns:
      whether this syntax rule has been parsed.

    • timelessSequence

      protected void timelessSequence(List<Timeless> target)
      Parser code (=test, consume, store to target, and return flag) for one timeless item, whch can be a value or an assigment. FIXME STIMMT NICHT MERH: timeless ::= ident = token_or_list
      token_or_list ::= "(" token_or_list* ")" | non_white_space
      FIXME IMPLEMENT FIXME IMPLEMENT Zugriffsfunktionen ("asList" "asInt" "asString" etc.) in base/Util. 
          CONFORM x
    a = b
      VOX c
       a = b
      VOX d
       a = b
    PARS a
       a = b
    PARS b
    a = b
       VOX c
       a = b
       T        17      18
       VOX c    x   x   x
      "VOX" "PARS" and "T" stop accepting.
      Parameters:
      target - where to store the parsed/constructed data

    • timelessLine

      protected Timeless timelessLine()
      Parses a = b = X and a = b = (c=d e=f 1 2 3)

    • timelessSingleValue

      @Opt protected @Opt String timelessSingleValue()
      Accept all tokens which are not whitespace or parantheses and concatenate their contents. "+-"3 = value ist möglich !?!?!? "" = value ist möglich !?!?!?

    • voxParams

      protected boolean voxParams(Container container)
      Parser function (=test, consume and return) for
      voxParams ::= VOX_and_name conform?, timeless* Creates a new voice by calling VOX_and_name(Container,boolean), which stores this either to the timescape or to the part under construction. Each voice name may only appear once in these declarations.
      Returns:
      whether this syntax rule has been parsed.

    • part

      protected boolean part(TimeScape current_ts)
      Parser function (=test, consume and return) for
      pars ::= PARS ident_or_stringconst conform timeless* voiceparams accolade*
      Parameters:
      current_ts - where to store the constructed PARS data
      Returns:
      whether this syntax rule has been parsed.

    • accolade

      protected boolean accolade(Part current_part)
      Parser function (=test, consume and return) for
      accolade ::= T timeline NL vox* This is the central parsing function which creates time points, events, voices and registers all these in the part under constrction.
      Top time points may use more than one (>1) source line, iff allowed by modifiers. The possible characters for sub-divisions are given as string constants in modifiers. Each source fragment not contained therein is assumed a top-level time point source and stored for LATER parsing in Part.tlines.
      Enter resulting timeline map into "current_part" argument, in field ".tlines", and event data in the voice objects, entered also in "current_part".
      Returns:
      whether this syntax rule has been parsed.

    • makeLowerTps

      protected void makeLowerTps(SortedMap<Integer,Tp> col2tp, String sign, Map<Integer,String> sources, Map<Integer,List<Token>> unpacked)
      Step through T-line source fragments and create TpSubs which divide the interval between already recognized, higher time point levels.
      Parameters:
      col2tp - In and OUTPUT parameter where to store the new column definitions.
      sign - the single string which (as only!) stands for division points of this level.
      sources - the source fragment sequence, packed into one(1) string for each position=column.
      unpacked - the original source token sequence, only needed to get location information for messages

    • VOX_and_name

      @Opt protected @Opt Vox VOX_and_name(Container container, boolean multiple)
      Parse function (=test, consume and return) for
      VOX_and_name ::= VOX ident
      Can be called (a) in context of meta values (in a score or in a part, before the first T-line) or (b) in an accolade after the T-line. In (a) the voice is only carrier of timeless attributes, which later can be (possibly) distributed to voices with the same name in accolades. No voice name may be repeated.

      In (b) it is contained in a part and may be repeated, but not in the same accolade.

      If the Vox object dos not yet exists: create it and store it to score or part.

      Returns:
      whether this syntax rule has been parsed.

    • voice

      boolean voice(Part part, SortedMap<Integer,Tp> col2tp, Set<String> voicesInAccolade)
      Parse function (=test, consume and return) for
      "vox ::= VOX ident token NL (paramIdent token* NL)"
      Is called in the context of parsing an accolade, i.e. events under a T-line. Store resulting vox object to argument "part", and resulting events to the vox object (into several different collections in parallel). Generates new timepoints iff events start on columns not defined by the T-line.
      Returns:
      whether this syntax rule has been parsed.

    • foldInto

      protected SortedMap<Integer,List<Token>> foldInto(SortedMap<Integer,RawParser.PosList> target, SortedMap<Integer,List<Token>> data, boolean isTopLevel)
      Folds a sequence of input tokens in the main parameter line of a voice into a sub-tree of RawParser.PosList, controled by opening and closing parentheses. As a first step, this is independent from the time points in the time line. Stores also the position of the closing parentheses of the level it is called for, see RawParser.PosList for details.
      Parameters:
      target - the pre-constructed output storage.
      data - input data, starting at this level

    • isTopTpText

      protected boolean isTopTpText(String t)
      Allow everything as top time point which is not contained in the string constants which make up the sub-dividers.

    • collectFragments

      protected SortedMap<Integer,List<Token>> collectFragments(int firstFragmentColumn, boolean breakAtParentheses)
      Parser function (=test, consume and return) for unparsed, raw token sequence in time lines and parameter lines. Collects all contiguous non-white-space fragments ordered by increasing starting column, and (secondarily) by increasing line numbers; returns them in a SortedMap.
      New fragment is started with whitespace, newline, etc.
      Double quotes are used to escape whitespace and parentheses break behaviour; quote signs are stripped from result.
      Stops whenever the offside given by firstFragmentColumn is undercut.
      GLO IN nextToken, consume()
      Parameters:
      breakAtParentheses - whether parentheses are always a fragment on their own.
      Returns:
      a map from the column numbers to the sequence of all tokens starting there (in line order of the lines considered)

    • flatten

      protected SortedMap<Integer,String> flatten(Map<Integer,List<Token>> tokens, String separator)
      Concatenate source text of time point values which use more than one (>1) lines of input. Cf. Util/getParamText()

    • singleCheck

      protected SortedMap<Integer,String> singleCheck(Map<Integer,List<Token>> tokens)
      Check whether each collected fragment uses only one(1) input line, and return this only text as a (trivial) concatenation result.