|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object com.topologi.diffx.load.text.TokenizerByText
public final class TokenizerByText
The tokeniser for characters events.
This class is not synchronized.
Constructor Summary | |
---|---|
TokenizerByText(WhiteSpaceProcessing whitespace)
Creates a new tokenizer. |
Method Summary | |
---|---|
TextGranularity |
granurality()
Always TextGranularity.CHARACTER . |
List<TextEvent> |
tokenize(CharSequence seq)
Returns the list of TextEvent corresponding to the specified character sequence. |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public TokenizerByText(WhiteSpaceProcessing whitespace)
whitespace
- the whitespace processing for this tokenizer.
NullPointerException
- if the white space processing is not specified.Method Detail |
---|
public List<TextEvent> tokenize(CharSequence seq)
TextEvent
corresponding to the specified character sequence.
tokenize
in interface TextTokenizer
seq
- the character sequence to tokenize.
public TextGranularity granurality()
TextGranularity.CHARACTER
.
Returns the text granularity of this tokenizer.
granurality
in interface TextTokenizer
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |