org.eclipse.jdt.internal.compiler.parser
Class Scanner
java.lang.Object
org.eclipse.jdt.internal.compiler.parser.Scanner
- All Implemented Interfaces:
- TerminalTokens
- Direct Known Subclasses:
- RecoveryScanner
public class Scanner
- extends Object
- implements TerminalTokens
IMPORTANT NOTE: Internal Scanner implementation. It is mirrored in
org.eclipse.jdt.core.compiler public package where it is API.
The mirror implementation is using the backward compatible ITerminalSymbols constant
definitions (stable with 2.0), whereas the internal implementation uses TerminalTokens
which constant values reflect the latest parser generation state.
Fields inherited from interface org.eclipse.jdt.internal.compiler.parser.TerminalTokens |
TokenNameabstract, TokenNameAND, TokenNameAND_AND, TokenNameAND_EQUAL, TokenNameassert, TokenNameAT, TokenNameboolean, TokenNamebreak, TokenNamebyte, TokenNamecase, TokenNamecatch, TokenNamechar, TokenNameCharacterLiteral, TokenNameclass, TokenNameCOLON, TokenNameCOMMA, TokenNameCOMMENT_BLOCK, TokenNameCOMMENT_JAVADOC, TokenNameCOMMENT_LINE, TokenNameconst, TokenNamecontinue, TokenNamedefault, TokenNameDIVIDE, TokenNameDIVIDE_EQUAL, TokenNamedo, TokenNameDOT, TokenNamedouble, TokenNameDoubleLiteral, TokenNameELLIPSIS, TokenNameelse, TokenNameenum, TokenNameEOF, TokenNameEQUAL, TokenNameEQUAL_EQUAL, TokenNameERROR, TokenNameextends, TokenNamefalse, TokenNamefinal, TokenNamefinally, TokenNamefloat, TokenNameFloatingPointLiteral, TokenNamefor, TokenNamegoto, TokenNameGREATER, TokenNameGREATER_EQUAL, TokenNameIdentifier, TokenNameif, TokenNameimplements, TokenNameimport, TokenNameinstanceof, TokenNameint, TokenNameIntegerLiteral, TokenNameinterface, TokenNameLBRACE, TokenNameLBRACKET, TokenNameLEFT_SHIFT, TokenNameLEFT_SHIFT_EQUAL, TokenNameLESS, TokenNameLESS_EQUAL, TokenNamelong, TokenNameLongLiteral, TokenNameLPAREN, TokenNameMINUS, TokenNameMINUS_EQUAL, TokenNameMINUS_MINUS, TokenNameMULTIPLY, TokenNameMULTIPLY_EQUAL, TokenNamenative, TokenNamenew, TokenNameNOT, TokenNameNOT_EQUAL, TokenNamenull, TokenNameOR, TokenNameOR_EQUAL, TokenNameOR_OR, TokenNamepackage, TokenNamePLUS, TokenNamePLUS_EQUAL, TokenNamePLUS_PLUS, TokenNameprivate, TokenNameprotected, TokenNamepublic, TokenNameQUESTION, TokenNameRBRACE, TokenNameRBRACKET, TokenNameREMAINDER, TokenNameREMAINDER_EQUAL, TokenNamereturn, TokenNameRIGHT_SHIFT, TokenNameRIGHT_SHIFT_EQUAL, TokenNameRPAREN, TokenNameSEMICOLON, TokenNameshort, TokenNamestatic, TokenNamestrictfp, TokenNameStringLiteral, TokenNamesuper, TokenNameswitch, TokenNamesynchronized, TokenNamethis, TokenNamethrow, TokenNamethrows, TokenNametransient, TokenNametrue, TokenNametry, TokenNameTWIDDLE, TokenNameUNSIGNED_RIGHT_SHIFT, TokenNameUNSIGNED_RIGHT_SHIFT_EQUAL, TokenNamevoid, TokenNamevolatile, TokenNamewhile, TokenNameWHITESPACE, TokenNameXOR, TokenNameXOR_EQUAL |
Constructor Summary |
Scanner()
|
Scanner(boolean tokenizeComments,
boolean tokenizeWhiteSpace,
boolean checkNonExternalizedStringLiterals,
long sourceLevel,
char[][] taskTags,
char[][] taskPriorities,
boolean isTaskCaseSensitive)
|
Scanner(boolean tokenizeComments,
boolean tokenizeWhiteSpace,
boolean checkNonExternalizedStringLiterals,
long sourceLevel,
long complianceLevel,
char[][] taskTags,
char[][] taskPriorities,
boolean isTaskCaseSensitive)
|
sourceLevel
public long sourceLevel
complianceLevel
public long complianceLevel
useAssertAsAnIndentifier
public boolean useAssertAsAnIndentifier
containsAssertKeyword
public boolean containsAssertKeyword
useEnumAsAnIndentifier
public boolean useEnumAsAnIndentifier
recordLineSeparator
public boolean recordLineSeparator
currentCharacter
public char currentCharacter
startPosition
public int startPosition
currentPosition
public int currentPosition
initialPosition
public int initialPosition
eofPosition
public int eofPosition
skipComments
public boolean skipComments
tokenizeComments
public boolean tokenizeComments
tokenizeWhiteSpace
public boolean tokenizeWhiteSpace
source
public char[] source
withoutUnicodeBuffer
public char[] withoutUnicodeBuffer
withoutUnicodePtr
public int withoutUnicodePtr
unicodeAsBackSlash
public boolean unicodeAsBackSlash
scanningFloatLiteral
public boolean scanningFloatLiteral
COMMENT_ARRAYS_SIZE
public static final int COMMENT_ARRAYS_SIZE
- See Also:
- Constant Field Values
commentStops
public int[] commentStops
commentStarts
public int[] commentStarts
commentTagStarts
public int[] commentTagStarts
commentPtr
public int commentPtr
lastCommentLinePosition
protected int lastCommentLinePosition
foundTaskTags
public char[][] foundTaskTags
foundTaskMessages
public char[][] foundTaskMessages
foundTaskPriorities
public char[][] foundTaskPriorities
foundTaskPositions
public int[][] foundTaskPositions
foundTaskCount
public int foundTaskCount
taskTags
public char[][] taskTags
taskPriorities
public char[][] taskPriorities
isTaskCaseSensitive
public boolean isTaskCaseSensitive
diet
public boolean diet
lineEnds
public int[] lineEnds
linePtr
public int linePtr
wasAcr
public boolean wasAcr
END_OF_SOURCE
public static final String END_OF_SOURCE
- See Also:
- Constant Field Values
INVALID_HEXA
public static final String INVALID_HEXA
- See Also:
- Constant Field Values
INVALID_OCTAL
public static final String INVALID_OCTAL
- See Also:
- Constant Field Values
INVALID_CHARACTER_CONSTANT
public static final String INVALID_CHARACTER_CONSTANT
- See Also:
- Constant Field Values
INVALID_ESCAPE
public static final String INVALID_ESCAPE
- See Also:
- Constant Field Values
INVALID_INPUT
public static final String INVALID_INPUT
- See Also:
- Constant Field Values
INVALID_UNICODE_ESCAPE
public static final String INVALID_UNICODE_ESCAPE
- See Also:
- Constant Field Values
INVALID_FLOAT
public static final String INVALID_FLOAT
- See Also:
- Constant Field Values
INVALID_LOW_SURROGATE
public static final String INVALID_LOW_SURROGATE
- See Also:
- Constant Field Values
INVALID_HIGH_SURROGATE
public static final String INVALID_HIGH_SURROGATE
- See Also:
- Constant Field Values
NULL_SOURCE_STRING
public static final String NULL_SOURCE_STRING
- See Also:
- Constant Field Values
UNTERMINATED_STRING
public static final String UNTERMINATED_STRING
- See Also:
- Constant Field Values
UNTERMINATED_COMMENT
public static final String UNTERMINATED_COMMENT
- See Also:
- Constant Field Values
INVALID_CHAR_IN_STRING
public static final String INVALID_CHAR_IN_STRING
- See Also:
- Constant Field Values
INVALID_DIGIT
public static final String INVALID_DIGIT
- See Also:
- Constant Field Values
INVALID_BINARY
public static final String INVALID_BINARY
- See Also:
- Constant Field Values
BINARY_LITERAL_NOT_BELOW_17
public static final String BINARY_LITERAL_NOT_BELOW_17
- See Also:
- Constant Field Values
ILLEGAL_HEXA_LITERAL
public static final String ILLEGAL_HEXA_LITERAL
- See Also:
- Constant Field Values
INVALID_UNDERSCORE
public static final String INVALID_UNDERSCORE
- See Also:
- Constant Field Values
UNDERSCORES_IN_LITERALS_NOT_BELOW_17
public static final String UNDERSCORES_IN_LITERALS_NOT_BELOW_17
- See Also:
- Constant Field Values
OptimizedLength
public static final int OptimizedLength
- See Also:
- Constant Field Values
charArray_length
public final char[][][][] charArray_length
TAG_PREFIX
public static final char[] TAG_PREFIX
TAG_PREFIX_LENGTH
public static final int TAG_PREFIX_LENGTH
TAG_POSTFIX
public static final char TAG_POSTFIX
- See Also:
- Constant Field Values
TAG_POSTFIX_LENGTH
public static final int TAG_POSTFIX_LENGTH
- See Also:
- Constant Field Values
nlsTagsPtr
protected int nlsTagsPtr
checkNonExternalizedStringLiterals
public boolean checkNonExternalizedStringLiterals
lastPosition
protected int lastPosition
returnOnlyGreater
public boolean returnOnlyGreater
insideRecovery
public boolean insideRecovery
RoundBracket
public static final int RoundBracket
- See Also:
- Constant Field Values
SquareBracket
public static final int SquareBracket
- See Also:
- Constant Field Values
CurlyBracket
public static final int CurlyBracket
- See Also:
- Constant Field Values
BracketKinds
public static final int BracketKinds
- See Also:
- Constant Field Values
LOW_SURROGATE_MIN_VALUE
public static final int LOW_SURROGATE_MIN_VALUE
- See Also:
- Constant Field Values
HIGH_SURROGATE_MIN_VALUE
public static final int HIGH_SURROGATE_MIN_VALUE
- See Also:
- Constant Field Values
HIGH_SURROGATE_MAX_VALUE
public static final int HIGH_SURROGATE_MAX_VALUE
- See Also:
- Constant Field Values
LOW_SURROGATE_MAX_VALUE
public static final int LOW_SURROGATE_MAX_VALUE
- See Also:
- Constant Field Values
Scanner
public Scanner()
Scanner
public Scanner(boolean tokenizeComments,
boolean tokenizeWhiteSpace,
boolean checkNonExternalizedStringLiterals,
long sourceLevel,
long complianceLevel,
char[][] taskTags,
char[][] taskPriorities,
boolean isTaskCaseSensitive)
Scanner
public Scanner(boolean tokenizeComments,
boolean tokenizeWhiteSpace,
boolean checkNonExternalizedStringLiterals,
long sourceLevel,
char[][] taskTags,
char[][] taskPriorities,
boolean isTaskCaseSensitive)
atEnd
public final boolean atEnd()
checkTaskTag
public void checkTaskTag(int commentStart,
int commentEnd)
throws InvalidInputException
- Throws:
InvalidInputException
getCurrentIdentifierSource
public char[] getCurrentIdentifierSource()
getCurrentTokenEndPosition
public int getCurrentTokenEndPosition()
getCurrentTokenSource
public char[] getCurrentTokenSource()
getCurrentTokenString
public final String getCurrentTokenString()
getCurrentTokenSourceString
public char[] getCurrentTokenSourceString()
getCurrentStringLiteral
public final String getCurrentStringLiteral()
getRawTokenSource
public final char[] getRawTokenSource()
getRawTokenSourceEnd
public final char[] getRawTokenSourceEnd()
getCurrentTokenStartPosition
public int getCurrentTokenStartPosition()
getLineEnd
public final int getLineEnd(int lineNumber)
getLineEnds
public final int[] getLineEnds()
getLineStart
public final int getLineStart(int lineNumber)
- Search the source position corresponding to the beginning of a given line number
Line numbers are 1-based, and relative to the scanner initialPosition.
Character positions are 0-based.
e.g. getLineStart(1) --> 0 indicates that the first line starts at character 0.
In case the given line number is inconsistent, answers -1.
- Parameters:
lineNumber
- int
- Returns:
- int
getNextChar
public final int getNextChar()
getNextCharWithBoundChecks
public final int getNextCharWithBoundChecks()
getNextChar
public final boolean getNextChar(char testedChar)
getNextChar
public final int getNextChar(char testedChar1,
char testedChar2)
getNextCharAsDigit
public final boolean getNextCharAsDigit()
throws InvalidInputException
- Throws:
InvalidInputException
getNextCharAsDigit
public final boolean getNextCharAsDigit(int radix)
getNextCharAsJavaIdentifierPartWithBoundCheck
public boolean getNextCharAsJavaIdentifierPartWithBoundCheck()
getNextCharAsJavaIdentifierPart
public boolean getNextCharAsJavaIdentifierPart()
scanIdentifier
public int scanIdentifier()
throws InvalidInputException
- Throws:
InvalidInputException
getNextToken
public int getNextToken()
throws InvalidInputException
- Throws:
InvalidInputException
getNextUnicodeChar
public void getNextUnicodeChar()
throws InvalidInputException
- Throws:
InvalidInputException
getNLSTags
public NLSTag[] getNLSTags()
getSource
public char[] getSource()
isFirstTag
protected boolean isFirstTag()
jumpOverMethodBody
public final void jumpOverMethodBody()
jumpOverUnicodeWhiteSpace
public final boolean jumpOverUnicodeWhiteSpace()
throws InvalidInputException
- Throws:
InvalidInputException
pushLineSeparator
public final void pushLineSeparator()
pushUnicodeLineSeparator
public final void pushUnicodeLineSeparator()
recordComment
public void recordComment(int token)
resetTo
public void resetTo(int begin,
int end)
- Reposition the scanner on some portion of the original source. The given endPosition is the last valid position.
Beyond this position, the scanner will answer EOF tokens (
ITerminalSymbols.TokenNameEOF
).
- Parameters:
begin
- the given start positionend
- the given end position
scanEscapeCharacter
protected final void scanEscapeCharacter()
throws InvalidInputException
- Throws:
InvalidInputException
scanIdentifierOrKeywordWithBoundCheck
public int scanIdentifierOrKeywordWithBoundCheck()
scanIdentifierOrKeyword
public int scanIdentifierOrKeyword()
scanNumber
public int scanNumber(boolean dotPrefix)
throws InvalidInputException
- Throws:
InvalidInputException
getLineNumber
public final int getLineNumber(int position)
- Search the line number corresponding to a specific position
- Parameters:
position
- int
- Returns:
- int
setSource
public final void setSource(char[] sourceString)
setSource
public final void setSource(char[] contents,
CompilationResult compilationResult)
setSource
public final void setSource(CompilationResult compilationResult)
toString
public String toString()
- Overrides:
toString
in class Object
toStringAction
public String toStringAction(int act)
unicodeInitializeBuffer
public void unicodeInitializeBuffer(int length)
unicodeStore
public void unicodeStore()
unicodeStore
public void unicodeStore(char character)
isIdentifier
public static boolean isIdentifier(int token)
isLiteral
public static boolean isLiteral(int token)
isKeyword
public static boolean isKeyword(int token)
Copyright © 2012. All Rights Reserved.