com.shakelang.util.parseutils/com.shakelang.util.parseutils.lexer.token.stream/TokenInputStream

TokenInputStream

interface TokenInputStream<TT : TokenType, T : Token<TT>>(source)

A TokenInputStream provides the Tokens for a Parser. It is created by a lexer

Since

0.1.0

Inheritors

OnDemandLexingTokenInputStream

TokenBasedTokenInputStream

Properties

actual

abstract val actual: T

Returns the actual Token

actualEnd

open val actualEnd: Int

Returns the end of the actual token

actualHasValue

open val actualHasValue: Boolean

Checks if the actual token without changing the actual token

actualStart

open val actualStart: Int

Returns the start of the actual token

actualType

open val actualType: TT

Returns the type of the actual token

actualValue

open val actualValue: String?

Returns the value of the actual

map

abstract val map: PositionMap

The map for the token-positions We have this map to resolve the column / line of an index. This is useful for error-generation.

position

abstract val position: Int

The position that the TokenInputStream is actually at

size

abstract val size: Int

The size of the TokenInputStream

source

abstract val source: String

The source (mostly filename) of the TokenInputStream

Functions

has

abstract fun has(num: Int): Boolean

Checks if the TokenInputStream has left a given number of tokens

hasNext

open operator fun hasNext(): Boolean

Checks if the TokenInputStream has a token left

open operator fun next(): T

Returns the next token of the TokenInputStream (and skips)

nextType

open fun nextType(): TT

Returns the type of the next token of the TokenInputStream (and skips)

nextValue

open fun nextValue(): String?

Returns the next token of the TokenInputStream

peek

open fun peek(): T

Returns the next Token

abstract fun peek(offset: Int): T

Peek the token at the given index

peekEnd

open fun peekEnd(): Int

Returns the end of the next token without changing the actual token

open fun peekEnd(offset: Int): Int

Peek the token end at the given index

peekHasValue

open fun peekHasValue(): Boolean

Checks if the next token of the TokenInputStream has a value without skipping

open fun peekHasValue(offset: Int): Boolean

Checks if the token at the given offset of the TokenInputStream has a value without changing the actual token

peekStart

open fun peekStart(): Int

Returns the start of the next token without changing the actual token

open fun peekStart(offset: Int): Int

Peek the token start at the given index

peekType

open fun peekType(): TT

Returns the type of the next token without changing the actual token

open fun peekType(offset: Int): TT

Peek the token type at the given index

peekValue

open fun peekValue(): String?

Returns the value of the next token without changing the actual token

open fun peekValue(offset: Int): String?

Peek the token value at the given index

skip

abstract fun skip()

Skips the next token

abstract fun skip(amount: Int)

Skips a number of tokens