Splits a StringView in tokens according to separators.
More...
#include <StringView.h>
Splits a StringView in tokens according to separators.
◆ Options
Enumerator |
---|
IncludeEmpty | If to tokenizeNext should return also empty tokens.
|
SkipEmpty | If to tokenizeNext should NOT return also empty tokens.
|
◆ StringViewTokenizer()
SC::StringViewTokenizer::StringViewTokenizer |
( |
StringView |
text | ) |
|
|
inline |
Build a tokenizer operating on the given text string view.
◆ countTokens()
Count the number of tokens that exist in the string view passed in constructor, when splitted along the given separators.
- Parameters
-
separators | Separators to split the original string with |
- Returns
- Current StringViewTokenizer to inspect SC::StringViewTokenizer::numSplitsNonEmpty or SC::StringViewTokenizer::numSplitsTotal.
Example:
#define SC_TEST_EXPECT(e)
Records a test expectation (eventually aborting or breaking o n failed test)
Definition: Testing.h:113
StringViewTokenizer(StringView text)
Build a tokenizer operating on the given text string view.
Definition: StringView.h:589
StringViewTokenizer & countTokens(Span< const StringCodePoint > separators)
Count the number of tokens that exist in the string view passed in constructor, when splitted along t...
size_t numSplitsNonEmpty
How many non-empty splits have occurred in current tokenization.
Definition: StringView.h:575
size_t numSplitsTotal
How many total splits have occurred in current tokenization.
Definition: StringView.h:576
◆ isFinished()
bool SC::StringViewTokenizer::isFinished |
( |
| ) |
const |
Check if the tokenizer has processed the entire the string view passed in the constructor.
◆ tokenizeNext()
Splits the string along a list of separators.
- Parameters
-
separators | List of separators |
options | If to skip empty tokens or not |
- Returns
true
if there are additional tokens to parse
Example:
{
console.printLine(tokenizer.component);
}
@ SkipEmpty
If to tokenizeNext should NOT return also empty tokens.
Definition: StringView.h:585
◆ tokenizeNextLine()
bool SC::StringViewTokenizer::tokenizeNextLine |
( |
| ) |
|
|
inline |
Tokenizes from current position to first newline.
- Returns
true
if a new line has been found
Example:
◆ component
Current component that has been tokenized by tokenizeNext.
◆ numSplitsNonEmpty
size_t SC::StringViewTokenizer::numSplitsNonEmpty = 0 |
How many non-empty splits have occurred in current tokenization.
◆ numSplitsTotal
size_t SC::StringViewTokenizer::numSplitsTotal = 0 |
How many total splits have occurred in current tokenization.
◆ processed
Substring of original string passed in constructor processed so far.
◆ remaining
Substring from current position until the end of original text.
◆ splittingCharacter
The last splitting character matched in current tokenization.
The documentation for this struct was generated from the following file: