new Tokenizer(proto)
Constructs a new Tokenizer.
Parameters:
Name | Type | Description |
---|---|---|
proto |
string | Proto to tokenize |
- Source:
Members
-
index :number
-
Current index.
Type:
- number
- Source:
-
line :number
-
Current line.
Type:
- number
- Source:
-
readingString :boolean
-
Whether currently reading a string or not.
Type:
- boolean
- Source:
-
source :string
-
Source to parse.
Type:
- string
- Source:
-
stack :Array
-
Stacked values.
Type:
- Array
- Source:
-
stringEndsWith :string
-
Whatever character ends the string. Either a single or double quote character.
Type:
- string
- Source:
Methods
-
next() → (nullable) {string}
-
Gets the next token and advances by one.
- Source:
Throws:
-
If it's not a valid proto file
- Type
- Error
Returns:
Token or
null
on EOF- Type
- string
-
peek() → (nullable) {string}
-
Peeks for the next token.
- Source:
Throws:
-
If it's not a valid proto file
- Type
- Error
Returns:
Token or
null
on EOF- Type
- string
-
toString() → {string}
-
Returns a string representation of this object.
- Source:
Returns:
String representation as of "Tokenizer(index/length)"
- Type
- string