|
AsString
|
The AsString property is a read-only property which returns the string parsed from the stream. Usually, this is the exact string parsed from the stream. However, if the parser chooses to, the string returned could be a beautified version of the string. The intent is that the string should be reparsable by the parser to create the exact same token. Also, usually this property will not return the CommonInterfaces.IBaseStreamToken.Whitespace found before this token was parsed from the stream.
|
|
DataType
|
The DataType property provides a place where the data type of the string token can be set and retrieved from. Since the exact data types a token may take is specific to a parser, the DataType property is an integer. Parsers need to define integer constants or an enumeration which define the all unique types the token can take.
When the parser sets a token to be a numeric type, it should also place the numeric value in the CommonInterfaces.IBaseStreamToken.Number property.
|
|
LineNumber
|
The LineNumber property is intended to hold the line number where the token was found in the parsed stream.
|
|
Number
|
The Number property should hold the numeric value of the string parsed from the stream. When a numeric value (either an integer or float) is found in a stream, the parser creates a token and sets this property to the value parsed. It also should set the CommonInterfaces.IBaseStreamToken.DataType property to the correct numeric data type.
 Note
|
Setting the Number property should not set the DataType property unless there is only one numeric type for that parser. Usually, there are at least two numeric types (integer and float) so the DataType should be set separately from the Number property.
|
|
|
Text
|
The Text property should hold the string representation of the string parsed from the stream. Note that this text string is not necessarily the actual string read from the stream. For example, when RiverSoftAVG's parsers encounter the end of file/stream character, they fill in this Text property not with the actual characters but with a constant 'EOF' string. When a string value is found in a stream, the parser creates a token and sets this property to the value parsed. It also should set the CommonInterfaces.IBaseStreamToken.DataType property to the correct string data type.
|
|
Whitespace
|
The Whitespace property is intended to hold the white space (spaces, tabs, carraige returns, etc) that was found in the stream from the last token parsed to the current token. The decision of what is white space is up to the parser.
|
