33
34
* A superclass for all tokenizer algorithms.
35
36
* @author FracPete (fracpete at waikato dot ac dot nz)
38
39
public abstract class Tokenizer
40
implements Enumeration, OptionHandler, Serializable, RevisionHandler {
42
43
* Returns a string describing the stemmer