Package com.wcohen.ss.tokens

Class Summary
BasicToken An interned version of a string.
NGramTokenizer Wraps another tokenizer, and adds all computes all ngrams of characters from a single token produced by the inner tokenizer.
SimpleTokenizer Simple implementation of a Tokenizer.
TagLinkToken