Token Class Reference

Represents a single unit of a command language. More...

#include <Token.h>

Public Member Functions
	Token (const std::string &rawToken, const char precedingDelim, const char followingDelim)
	Creates a Token with the given content and surrounding delimiters. More...
const std::string &	getToken () const
	Gets the content of the Token. More...
const std::string &	finalize ()
	Removes any Tokenizer-specific formatting. More...
const char	getPrecedingDelimiter () const
	Gets the delimiter that came before this Token in the source string. More...
const char	getFollowingDelimiter () const
	Gets the delimiter that came after this Token in the source string. More...
bool	isToken (const char delim) const
	Checks if this Token instance is also a Token for the given delimiter. More...
bool	operator== (const Token &other) const
	Checks for equality between tokens. More...

Static Public Member Functions
static bool	isToken (const std::string &candidate, const char delim)
	Checks if the given candidate string is a Token for the given delimiter. More...

Detailed Description

Represents a single unit of a command language.

Stores information about the delimiters on either side of the Token from its source in case the delimiters provide contextual information about the token.

There is usually no need to construct a Token directly, use a Tokenizer to get Tokens from an input string.

Since

0.5

Constructor & Destructor Documentation

Token()

Token::Token	(	const std::string &	rawToken,
		const char	precedingDelim,
		const char	followingDelim
	)

Creates a Token with the given content and surrounding delimiters.

If a Token does not have a delimiter on a side (possibly due to it being at the beginning or end of input), the null character (`'\0'`) should be passed in as appropriate.

Parameters

rawToken	the content of the Token
precedingDelim	the delimiter preceding the token, or `'\0'` if not applicable
followingDelim	the delimiter following the token, or `'\0'` if not applicable

Member Function Documentation

finalize()

const string & Token::finalize

(

)

Removes any Tokenizer-specific formatting.

Specifically, this removes backslashes and leading/trailing whitespace.

Returns

the finalized Token

getFollowingDelimiter()

const char Token::getFollowingDelimiter

(

)

const

Gets the delimiter that came after this Token in the source string.

Returns

the following delimiter

getPrecedingDelimiter()

const char Token::getPrecedingDelimiter

(

)

const

Gets the delimiter that came before this Token in the source string.

Returns

the preceding delimiter

getToken()

const string & Token::getToken

(

)

const

Gets the content of the Token.

Returns

the content of the Token

isToken() [1/2]

bool Token::isToken

(

const char

delim

)

const

Checks if this Token instance is also a Token for the given delimiter.

Parameters

delim

the delimiter to check for

Returns

true if no instances of the delimiter were found (excluding leading/trailing delimiters), false otherwise

isToken() [2/2]

static bool Token::isToken ( const std::string &  candidate, const char  delim  )

static

Checks if the given candidate string is a Token for the given delimiter.

Parameters

candidate	the candidate string to search through
delim	the delimiter to check for

Returns

true if no instances of the delimiter were found (excluding leading/trailing delimiters), false otherwise

operator==()

bool Token::operator==

(

const Token &

other

)

const

Checks for equality between tokens.

This does not account for the delimiters on either side of the token.

Parameters

other

the other token to check for equality

Returns

true if the contents of the Tokens are equal, false otherwise.

---

The documentation for this class was generated from the following files:

• Token.h
• Token.cpp