Core Library: Core::Tokeniser Class Reference

Core Library

A class to split a string into separate tokens. More...

#include <Tokeniser.hpp>

List of all members.

Public Types
enum	Flags { NONE = 0x0000, MERGE_SEPS = 0x0001, RETURN_SEPS = 0x0002 }
	The flags that control the tokensiation. More...
typedef std::vector< tstring >	Tokens
	An array of strings.
Public Member Functions
	Tokeniser (const tstring &string, const tstring &seps, int flags=NONE)
	Construction from a string, separator list and flags.
	~Tokeniser ()
	Destructor.
bool	moreTokens () const
	Query if we've reached the end.
tstring	nextToken ()
	Get the next token.
Static Public Member Functions
static size_t	split (const tstring &string, const tstring &seps, Tokens &tokens, uint flags=NONE)
	Tokenise the string into an array of strings.
Private Types
enum	TokenType { END_TOKEN = -1, VALUE_TOKEN = 1, SEPARATOR_TOKEN = 2 }
	The token types. More...
Private Attributes
const tstring &	m_string
	The string to tokenise.
const tstring &	m_seps
	The list of separators.
uint	m_flags
	The tokenising control flags.
TokenType	m_nextToken
	The next token type expected.
tstring::const_iterator	m_iter
	The string iterator.

Detailed Description

A class to split a string into separate tokens.

Member Typedef Documentation

typedef std::vector<tstring> Core::Tokeniser::Tokens

An array of strings.

Member Enumeration Documentation

enum Core::Tokeniser::Flags

The flags that control the tokensiation.

Enumerator:

NONE	Default.
MERGE_SEPS	Merge consecutive separators.
RETURN_SEPS	Return separators as tokens.

enum Core::Tokeniser::TokenType [private]

The token types.

Enumerator:

END_TOKEN	Reached end of tokens.
VALUE_TOKEN	Parsing value token.
SEPARATOR_TOKEN	Parsing separator list token.

Constructor & Destructor Documentation

Core::Tokeniser::Tokeniser	(	const tstring &	string,
		const tstring &	seps,
		int	flags = `NONE`
	)

Construction from a string, separator list and flags.

Core::Tokeniser::~Tokeniser ( )

Destructor.

Member Function Documentation

bool Core::Tokeniser::moreTokens ( ) const

Query if we've reached the end.

tstring Core::Tokeniser::nextToken ( )

Get the next token.

size_t Core::Tokeniser::split	(	const tstring &	string,
		const tstring &	seps,
		Tokens &	tokens,
		uint	flags = `NONE`
	)		`[static]`

Tokenise the string into an array of strings.

Member Data Documentation

const tstring& Core::Tokeniser::m_string [private]

The string to tokenise.

const tstring& Core::Tokeniser::m_seps [private]

The list of separators.

uint Core::Tokeniser::m_flags [private]

The tokenising control flags.

TokenType Core::Tokeniser::m_nextToken [private]

The next token type expected.

tstring::const_iterator Core::Tokeniser::m_iter [private]

The string iterator.

The documentation for this class was generated from the following files: