When you created a custom TokenizerFactory for Apache Lucene, you need to register it so the SPI Loader can recognize it.
Let's assume, your have written a TokenizerFactory class like this:
package de.company.subdomain;
public class AwesomeTokenizerFactory extends TokenizerFactory {
// Lot of awesome code
}