Nowadays most free-text searching is based on Lucene-like approaches, where the search text is parsed into its -various components. For every keyword a lookup is done to see where it occurs. When looking for a couple of keywords -this approach is great. But what about it if you are not looking for just a couple of keywords, but a 100,000 of -them? Like, for example, checking against a dictionary?
+Nowadays most free-text searching is based on Lucene-like approaches, where the search text is parsed into its + various components. For every keyword a lookup is done to see where it occurs. When looking for a couple of keywords + this approach is great. But what about it if you are not looking for just a couple of keywords, but a 100,000 of + them? Like, for example, checking against a dictionary?
-This is where the Aho-Corasick algorithm shines. Instead of chopping up the search text, it uses all the keywords -to build up a construct called a Trie. There are three crucial components -to Aho-Corasick:
+This is where the Aho-Corasick algorithm shines. Instead of chopping up the search text, it uses all the keywords + to build up a construct called a Trie. There are three crucial components + to Aho-Corasick:
-Every character encountered is presented to a state object within the goto structure. If there is a matching state, -that will be elevated to the new current state.
+Every character encountered is presented to a state object within the goto structure. If there is a matching state, + that will be elevated to the new current state.
-However, if there is no matching state, the algorithm will fall back to states with less depth (ie, a match less long) -and proceed from there, until it found a matching state, or it has reached the root state.
+However, if there is no matching state, the algorithm will signal a fail and fall back to states with less depth + (ie, a match less long) and proceed from there, until it found a matching state, or it has reached the root state.
-Whenever a state is reached that matches an entire keyword, it is emitted to an output set which can be read after the -entire scan has completed.
+Whenever a state is reached that matches an entire keyword, it is emitted to an output set which can be read after + the entire scan has completed.
-The beauty of the algorithm is that it is O(n). No matter how many keywords you have, or how big the search text is, -the performance will decline in a linear way.
+The beauty of the algorithm is that it is O(n). No matter how many keywords you have, or how big the search text is, + the performance will decline in a linear way.
-Some examples you could use the Aho-Corasick algorithm for:
+Some examples you could use the Aho-Corasick algorithm for:
-This library is the Java implementation of the afore-mentioned Aho-Corasick algorithm for efficient string matching. -The algorithm is explained in great detail in the white paper written by -Aho and Corasick.
+This library is the Java implementation of the afore-mentioned Aho-Corasick algorithm for efficient string matching. + The algorithm is explained in great detail in the white paper written by + Aho and Corasick: ftp://163.13.200.222/assistant/bearhero/prog/%A8%E4%A5%A6/ac_bm.pdf
-Setting up the Trie is a piece of cake:
+Setting up the Trie is a piece of cake:
- Trie trie = new Trie();
+ Trie trie = new Trie();
trie.addKeyword("hers");
trie.addKeyword("his");
trie.addKeyword("she");
@@ -92,13 +92,28 @@ The algorithm is explained in great detail in the white paper written by
Collection<Emit> emits = trie.parseText("ushers");
-You can now read the set. In this case it will find the following:
+ You can now read the set. In this case it will find the following:
-In normal situations you probably want to remove overlapping instances, retaining the longest and left-most + matches.
+ + Trie trie = new Trie().removeOverlaps();
+ trie.addKeyword("hot");
+ trie.addKeyword("hot chocolate");
+ Collection<Emit> emits = trie.parseText("hot chocolate");
+The removeOverlaps method tells the Trie to remove all overlapping matches. For this it relies on the following + conflict resolution rules: 1) longer matches prevail over shorter matches, 2) left-most prevails over right-most. + There is only one result now:
+ +Licensed under the Apache License, Version 2.0 (the "License");