From 571daa895f0050689b297a0e63f6d9ee77665dab Mon Sep 17 00:00:00 2001 From: robert-bor Date: Fri, 31 Jan 2014 20:58:17 +0100 Subject: [PATCH] updated aho-corasick site --- index.html | 89 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 52 insertions(+), 37 deletions(-) diff --git a/index.html b/index.html index edb1904..fa0d31a 100644 --- a/index.html +++ b/index.html @@ -44,47 +44,47 @@

Introduction

-

Nowadays most free-text searching is based on Lucene-like approaches, where the search text is parsed into its -various components. For every keyword a lookup is done to see where it occurs. When looking for a couple of keywords -this approach is great. But what about it if you are not looking for just a couple of keywords, but a 100,000 of -them? Like, for example, checking against a dictionary?

+

Nowadays most free-text searching is based on Lucene-like approaches, where the search text is parsed into its + various components. For every keyword a lookup is done to see where it occurs. When looking for a couple of keywords + this approach is great. But what about it if you are not looking for just a couple of keywords, but a 100,000 of + them? Like, for example, checking against a dictionary?

-

This is where the Aho-Corasick algorithm shines. Instead of chopping up the search text, it uses all the keywords -to build up a construct called a Trie. There are three crucial components -to Aho-Corasick:

+

This is where the Aho-Corasick algorithm shines. Instead of chopping up the search text, it uses all the keywords + to build up a construct called a Trie. There are three crucial components + to Aho-Corasick:

-

Every character encountered is presented to a state object within the goto structure. If there is a matching state, -that will be elevated to the new current state.

+

Every character encountered is presented to a state object within the goto structure. If there is a matching state, + that will be elevated to the new current state.

-

However, if there is no matching state, the algorithm will fall back to states with less depth (ie, a match less long) -and proceed from there, until it found a matching state, or it has reached the root state.

+

However, if there is no matching state, the algorithm will signal a fail and fall back to states with less depth + (ie, a match less long) and proceed from there, until it found a matching state, or it has reached the root state.

-

Whenever a state is reached that matches an entire keyword, it is emitted to an output set which can be read after the -entire scan has completed.

+

Whenever a state is reached that matches an entire keyword, it is emitted to an output set which can be read after + the entire scan has completed.

-

The beauty of the algorithm is that it is O(n). No matter how many keywords you have, or how big the search text is, -the performance will decline in a linear way.

+

The beauty of the algorithm is that it is O(n). No matter how many keywords you have, or how big the search text is, + the performance will decline in a linear way.

-

Some examples you could use the Aho-Corasick algorithm for:

+

Some examples you could use the Aho-Corasick algorithm for:

-

This library is the Java implementation of the afore-mentioned Aho-Corasick algorithm for efficient string matching. -The algorithm is explained in great detail in the white paper written by -Aho and Corasick.

+

This library is the Java implementation of the afore-mentioned Aho-Corasick algorithm for efficient string matching. + The algorithm is explained in great detail in the white paper written by + Aho and Corasick: ftp://163.13.200.222/assistant/bearhero/prog/%A8%E4%A5%A6/ac_bm.pdf

-

-Usage

+

+ Usage

-

Setting up the Trie is a piece of cake:

+

Setting up the Trie is a piece of cake:

-
    Trie trie = new Trie();
+          
    Trie trie = new Trie();
     trie.addKeyword("hers");
     trie.addKeyword("his");
     trie.addKeyword("she");
@@ -92,13 +92,28 @@ The algorithm is explained in great detail in the white paper written by
     Collection<Emit> emits = trie.parseText("ushers");
 

 
-
You can now read the set. In this case it will find the following:

+          
You can now read the set. In this case it will find the following:

 
-
    
-
  • "she" at position 3
    • 
-
  • "he" at position 3
    • 
-
  • "hers" at position 5
    • 
-

+          
    
+              
  • "she" starting at position 1, ending at position 3
    • 
+              
  • "he" starting at position 2, ending at position 3
    • 
+              
  • "hers" starting at position 2, ending at position 5
    • 
+          
In normal situations you probably want to remove overlapping instances, retaining the longest and left-most
+          matches.

+
+          
    Trie trie = new Trie().removeOverlaps();
+    trie.addKeyword("hot");
+    trie.addKeyword("hot chocolate");
+    Collection<Emit> emits = trie.parseText("hot chocolate");
+

+
+          
The removeOverlaps method tells the Trie to remove all overlapping matches. For this it relies on the following
+              conflict resolution rules: 1) longer matches prevail over shorter matches, 2) left-most prevails over right-most.
+              There is only one result now:

+
+          
    
+              
  • "hot chocolate" starting at position 0, ending at position 12
    • 
+          

 License

 
 
Licensed under the Apache License, Version 2.0 (the "License");