BaseTokeniser Class

Constructor for the BaseTokeniser which takes in a TextReader that the Tokeniser will generate Tokens from.

Gets the Current Line in the Input Stream.

Gets the Current Position in the Input Stream.

Gets the End Line in the Input Stream of the current Token.

Gets the End Position in the Input Stream of the current Token.

Gets/Sets the Format that this Tokeniser is used for.

Gets whether the Tokeniser has backtracked.

Gets/Sets the Last Token Type.

Gets the current length of the Output Buffer.

Gets the Start Line in the Input Stream of the current Token.

Gets the Start Position in the Input Stream of the current Token.

Gets the value of the Output Buffer.

Allows you to Backtrack one character (and no more).

Consumes a single Character into the Output Buffer and increments the Position Counters.

Consumes a single Character into the Output Buffer and increments the Position Counters.

Consumes a New Line (which may be a single \n or \r or the two characters following each other).

Consumes a New Line (which may be a single \n or \r or the two characters following each other).

Helper function which discards White Space which the Tokeniser doesn't care about and increments position counters correctly.

Determines whether the specified object is equal to the current object.

Helper Function for generating Standardised Parser Errors.

Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection.

Serves as the default hash function.

Gets the Next available Token from the Input.

Gets the Type of the current instance.

Handles the complex escapes that can occur in a local name.

Handles the standard escapes supported in all the UTF-8 based RDF serializations.

Determines whether a given Character can be valid as a Hex Digit.

Creates a shallow copy of the current Object.

Peeks at the next Character.

Skips a single Character and increments the Position Counters.

Informs the Helper that you wish to start reading a new Token.

Returns a string that represents the current object.

Helper Function for generating Standardised Parser Errors about unexpected characters.

Helper Function for generating Standardised Parser Errors about unexpected end of input.

Helper Function for generating Standardised Parser Errors about unexpected new lines.

Helper Function for generating Standardised Parser Errors about unexpected tokens.