Packages

package lexer

Type Members

  1. class ANTLRLexerAdaptor extends LexerBase

    This is the adaptor class for implementations of com.intellij.lexer.Lexer backed by an ANTLR 4 lexer.

    This is the adaptor class for implementations of com.intellij.lexer.Lexer backed by an ANTLR 4 lexer. It supports any ANTLR 4 lexer that does not store extra information for use in custom actions. For lexers that do not store custom state information, this default implementation is sufficient. Otherwise, subclass and override: {#getInitialState} and {#getLexerState}.

    Intellij lexers need to track state as they must be able to restart lexing in the middle of the input buffer. From Intellij doc:

    "A lexer that can be used incrementally may need to return its state, which means the context corresponding to each position in a file. For example, a Java lexer could have separate states for top level context, comment context and string literal context. An important requirement for a syntax highlighting lexer is that its state must be represented by a single integer number returned from Lexer.getState(). That state will be passed to the Lexer.start() method, along with the start offset of the fragment to process, when lexing is resumed from the middle of a file."

    This implementation supports single- as well as multi-mode lexers.

  2. class ANTLRLexerState extends AnyRef

    This class stores the state of an ANTLR lexer, such that it can be applied back to the lexer instance at a later time.

    This class stores the state of an ANTLR lexer, such that it can be applied back to the lexer instance at a later time.

    The default implementation stores the following fields, which provides support for any ANTLR 4 single- or multi-mode lexer that does not rely on custom state information for semantic predicates, custom embedded actions, and/or overridden methods such as Lexer#nextToken or Lexer#emit.

    • Lexer#_mode: The current lexer mode.
    • Lexer#_modeStack: The current lexer mode stack.

    If your lexer requires additional information to be stored, this class must be extended in the following ways.

    • Override #apply to ensure that the additional state information is applied to the provided lexer instance.
    • Override #hashCodeImpl and #equals to ensure that the caching features provided are able to efficiently store the resulting state instances.
    Annotations
    @Immutable()
  3. class PSIElementTypeFactory extends AnyRef

    The factory that automatically maps all tokens and rule names into IElementType objects: TokenIElementType and RuleIElementType.

    The factory that automatically maps all tokens and rule names into IElementType objects: TokenIElementType and RuleIElementType.

    This caches all mappings for each Language that use this factory. I.e., it's not keeping an instance per plugin/Language.

  4. class PSITokenSource extends TokenSource

    Make a PsiBuilder look like a source of ANTLR tokens.

    Make a PsiBuilder look like a source of ANTLR tokens. PsiBuilder provides tokens created by the lexer created in ParserDefinition#createLexer(Project). This is the bridge between the ANTLR lexer and parser objects. Normally we just create a org.antlr.v4.runtime.CommonTokenStream but the IDE has control and asks our ParserDefinition for the lexer and parser. This is how we hook them together. When IDE ask ParserDefinition for a parser, we will create one of these attached to the PsiBuilder.

  5. class RuleIElementType extends IElementType

    Represents a specific ANTLR rule invocation in the language of the plug-in and is the intellij "token type" of an interior PSI tree node.

    Represents a specific ANTLR rule invocation in the language of the plug-in and is the intellij "token type" of an interior PSI tree node. The IntelliJ equivalent of ANTLR RuleNode.getRuleIndex() method or maybe RuleNode itself.

    Intellij Lexer token types are instances of IElementType. We differentiate between parse tree subtree roots and tokens with RuleIElementType and TokenIElementType.

  6. class TokenIElementType extends IElementType

    Represents a token in the language of the plug-in.

    Represents a token in the language of the plug-in. The "token type" of leaf nodes in jetbrains PSI tree. Corresponds to ANTLR's int token type. Intellij lexer token types are instances of IElementType:

    "Interface for token types returned from lexical analysis and for types of nodes in the AST tree."

    We differentiate between parse tree subtree roots and tokens with RuleIElementType and TokenIElementType, respectively.

Ungrouped