com.shakelang.util.parseutils/com.shakelang.util.parseutils.lexer.token

Package-level declarations

Types

Link copied to clipboard
open class Token<T : TokenType>(val type: T, val value: String?, val start: Int, val end: Int)

The input of the com.shakelang.shake.lexer.Lexer gets converted into Tokens. These get parsed by the parser

Link copied to clipboard
interface TokenType

These are the different types of tokens, that the lexer creates