<?xml version="1.0"?><!DOCTYPE article SYSTEM "/project/take/software/searchbench_offline_processing/paperxml_generator/aclextractor/src/python/../resource/dtd/paperxml.dtd"><article><header><firstpageheader><page local="1" global="48"/><title>Script Independent Word Spotting in Multilingual Documents</title><pubinfo>The 2nd International Workshop On "Cross Lingual Information Access" Addressing the Information Need of Multilingual Societies, 2008</pubinfo><author surname="Bhardwaj" givenname="Anurag"><org  name="University of Buffalo" country="USA" city="Buffalo"/></author><author surname="Jose" givenname="Damien"><org  name="University of Buffalo" country="USA" city="Buffalo"/></author><author surname="Govindaraju" givenname="Venu"><org  name="University of Buffalo" country="USA" city="Buffalo"/></author></firstpageheader><frontmatter><p>Script Independent Word Spotting in Multilingual Documents</p><p>Anurag Bhardwaj, Damien Jose and Venu Govindaraju Center for Unified Biometrics and Sensors (CUBS) University at Buffalo, State University of New York Amherst, New York 14228 {ab94,dsjose,govind}@cedar.buffalo.edu</p></frontmatter><abstract><i>This paper describes a method for script independent word spotting in multilingual handwritten and machine printed documents. The system accepts a query in the form of text from the user and returns a ranked list of word images from document image corpus based on similarity with the query word. The system is divided into two main components. The first component known as Indexer, per­forms indexing ofall word images present in the document image corpus. This is achieved by extracting Moment Based features from word images and storing them as in­dex. A template is generated for keyword spotting which stores the mapping </i><i>ofa</i><i> keyword string to its correspond­ing word image which is used for generating query fea­ture vector. The second component, Similarity Matcher, returns a ranked list ofword images which are most sim­ilar to the query based on a cosine similarity metric. A manual Relevance feedback is applied based on Rocchio's formula, which re-formulates the query vector to return an improved ranked listing ofword images. The perfor­mance ofthe system is seen to be superior on printed text than on handwritten text. Experiments are reported on documents of three different languages: English, Hindi and Sanskrit. For handwritten English, an average pre­cision of </i>67% <i>was obtained for </i>30 <i>query words. For ma­chine printed Hindi, an average precision of </i>71% <i>was ob­tained for </i>75 <i>query words and for Sanskrit, an average precision of </i>87% <i>with 100 queries was obtained.</i> Figure 1 : A Sample English Document - Spotted Query word shown in the bounding box. </abstract></header><body><section number="1" title="Introduction"><p>The vast amount of information available in the form of handwritten and printed text in different languages poses a great challenge to the task of effective information ex­traction. Research in this area has primarily focussed on OCR based solutions which are adequate for Roman Lan­guage (A sample English document is shown in Figure 1). However efficient solutions do not exist for scripts like Devanagari. One of the main reasons for this is lack of generalisation. OCR solutions tend to be specific to script type. Ongoing research continues to scale these methods to different types and font sizes. Furthermore, non-Latin scripts exhibit complex character classes (like in the San­skrit document shown in Figure 2) and poor quality doc­uments are common.</p><p>The notion of Word spotting [6] has been introduced as an alternative to OCR based solutions. It can be de­fined as an information retrieval task that finds all oc-curences of a typed query word in a set of handwritten<page local="2" global="49"/></p><doubt alpha="0.0" length="1" tooSmall="False" monospace="0.0">1</doubt><p>^r^rwsr^r%^rl^^ra ii?ii ^i^d^d^litmnsaji fo«iiR«iiRHfiH Hrrof^r: mil</p><p>i <b>I</b>^t^f      f<b>^Rt </b>dT^fi ii <b>3 </b>ii</p><figure caption="Figure 2: A Sample Sanskrit Document - Spotted Query word shown in the bounding box."></figure><p>or machine printed documents. While spotting words in English has been explored [3, 5, 4, 7, 11], generalising these approaches to multiple scripts is still an ongoing re­search task. Harish et.al [1] describe a 'Gradient, Struc­tural, Concavity' (GSC) based feature set for word spot­ting in multiple scripts. However, they do not report the average precision rate for all queries in their experimen­tal results which makes it difficult to estimate the perfor­mance of their methodology.</p><p>One important factor in finding a script independent solution to word spotting is use of image based features which are invariant to script type, image scale and trans­lations. This paper proposes the use of moment based features for spotting word images in different scripts. We describe a moment-function based feature extraction scheme and use the standard vector space model to repre­sent the word images. Similarity between the query fea­ture vector and the indexed feature set is computed using a cosine similarity metric. We also apply the Rocchio for­mula based Manual Relevance feedback to improve the ranking of results obtained. We evaluate the performance of our system by conducting experiments on document images of three different scripts: English, Hindi and San­skrit.</p><p>The organization of the rest of the paper is as fol­lows: Section 2 describes the previous work. Section 3 describes the theory of moment functions. Section 4 de­scribes indexing word images and feature extraction. Sec­tion 5 describes the Similarity Matching and Relevance Feedback method applied to re-rank results. Section 6 describes the experiments and results. Future work and conclusions are outlined in Section 7.</p></section><section number="2" title="Previous Work"><p>Spotting words in English has recently received consider­able attention. Manmatha et al. [7], have proposed a com­bination of feature sets well suited for this application. For finding similarity between a query word image and the document word image, Dynamic Time warping [8] is commonly used. Although the approach has been promis­ing with English handwritten documents, it does not gen­eralise well across scripts. For eg., presence of Shirorekha in Devanagari script (an example shown in Figure 3) ren­ders most of the profile based features ineffective. Also, DTW based approaches are slow. Approaches which use a filter based feature set [2], are efficient with uniform font size and type but are not able to handle font variations and translations.</p><p>Harish et al. [1] use a Gradient, Structural and Concav­ity (GSC) feature set which measures the image character­istics at local, intermediate and large scales. Features are extracted using a 4x8 sampling window to gather informa­tion locally. Since character segmentation points are not perfectly located, local information about stroke orienta­tion and image gradient is not sufficient to characterize the change in font scale and type. Moreover, presence of noise in small regions of the word image lead to inconsis­tency in the overall feature extraction process. The per­formance of their approach is presented in terms of per­centage of the number of times the correct match was re­turned, which does not capture the recall rate of system. For English word spotting, their results do not state the size of the dataset and precision recall values have been reported for only 4 query words. For Sanskrit word spot­ting, the total number of query words is not mentioned which makes understanding of precision recall curve dif­ficult. A comparison of their results against our proposed method is presented in section 6.</p></section><section number="3" title="Moment Functions"><p>Moments and functions of moments have been previously used to achieve an invariant representation of a two-dimensional image pattern [9].   Geometrical moments [9] have the desirable property of being invariant under the image translation, scale and stretching and squeezing in either X or Y direction.<page local="3" global="50"/> Mathematically, such affine transformations are of the form of X * = <i>aX </i>+ b , and Y* = cY + d [10]. Geometrical Moments (GM) of order (p + q) for a continuous image function f (x, y) are defined as :</p><figure caption="Figure 3: A Sample Hindi Document - Spotted Query words shown in the bounding box."></figure><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">Mpq</doubt><doubt alpha="0.0" length="2" tooSmall="False" monospace="0.0">/7</doubt><doubt alpha="61.1" length="18" tooSmall="False" monospace="0.0">xpyqf (x, y) dx dy</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(1)</doubt><p>where p, q = 0,1, 2,..., to. The above definition has the form of the projection of the function f (x,y) onto the mononomial xpyq. In our case, where the function f (x, y) has only two possible values of 0 and 1, the equation 1 reduces to :</p><p>Mpq = EE xP yq f (x,y)</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(2)</doubt><doubt alpha="66.7" length="3" tooSmall="False" monospace="0.0">X Y</doubt><p>where X and Y represent x, y coordinates of the image. The center ofgravity ofthe image has the coordinates :</p><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">10</doubt><doubt alpha="91.7" length="12" tooSmall="True" monospace="0.0">Moment Order</doubt><figure caption="Figure 4: Average Precision curve Vs Moment Order for a Hindi Image Subset."></figure><p>They are used to normalise the coordinates by setting:</p><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">x</doubt><doubt alpha="20.0" length="5" tooSmall="False" monospace="0.0">-,y -</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">ay</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(6)</doubt><p>Using the normalised values of coordinates as obtained in equation 6 , the moment equation is as follows :</p><doubt alpha="50.0" length="18" tooSmall="False" monospace="0.0">J2xJ2Y^*ny*)qf(^y)</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">Moo</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(7)</doubt><p>whichis invariantunderimage translationandscale trans­formations.</p><doubt alpha="25.0" length="12" tooSmall="False" monospace="0.0">_      MlQ _</doubt><doubt alpha="100.0" length="3" tooSmall="False" monospace="0.0">Mqi</doubt><doubt alpha="66.7" length="9" tooSmall="False" monospace="0.0">Mqo Mqq '</doubt><p>If we refer to the center of gravity as origin, we obtain :</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(3)</doubt><doubt alpha="60.0" length="5" tooSmall="False" monospace="0.0">Mpq =</doubt><doubt alpha="36.0" length="25" tooSmall="False" monospace="0.0">££(x - x)p(y - y)qf (x,y)</doubt><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(4)</doubt><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">XY</doubt><p>These moments are also referred to as Central Moments and can be expressed as a linear combination of Mjkand the moments of lower order. The variances of the moment are defined as :</p><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">ax</doubt><doubt alpha="46.2" length="13" tooSmall="False" monospace="0.0">[tiho~_[m~q~2</doubt><p>VMoo'CTw~ Vmoo</p><doubt alpha="0.0" length="3" tooSmall="False" monospace="0.0">(5)</doubt></section><section number="4" title="Feature Extraction and Indexing"><p>Feature extraction is preceeded by preprocessing of doc­uments prior to computing moment based functions. Firstly, the Horizontal Profile feature of the document im­age is used to segment into line images. Thereafter, Ver­tical Profile features of each line image is used to extract individual word images. The word images are normalised to equal height and width of 256 pixels.</p><p>Using equation 7, moments up to the 7th order are extracted from the normalised word images. A feature vector consisting of 30 moment values obtained is con­structed for each word image and stored in the main index.<page local="4" global="51"/> Experiments were conducted to determine the num­ber of orders up to which moments should be computed. As shown in Figure 4, average precision increases with the rise in moment orders (up to a threshold of 7 orders), after which the precision rate falls. This can be attributed to the nature of higher order Geometrical Moments which are prone to adding noise in the feature set and thereby re­duce the overall precision after a certain threshold. After the index has been constructed using the moment features, we create a template which keeps the mapping between a word image and its corresponding text. This template is used to generate a query word image corresponding to the query text input by the user. A similar feature extraction mechanism is performed on the query word image to ob­tain a query feature vector which is used to find the sim­ilarity between the query word image and all other word images present in the corpus.</p><doubt alpha="0.0" length="4" tooSmall="True" monospace="0.0">0.86</doubt><doubt alpha="0.0" length="4" tooSmall="True" monospace="0.0">0.84</doubt><doubt alpha="0.0" length="4" tooSmall="True" monospace="0.0">0.82</doubt><doubt alpha="0.0" length="4" tooSmall="True" monospace="0.0">0.76</doubt><doubt alpha="0.0" length="4" tooSmall="True" monospace="0.0">0.74</doubt><doubt alpha="0.0" length="4" tooSmall="True" monospace="0.0">0.72</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.7</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">4</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">6</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">8</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">12</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">14</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">16</doubt></section><section number="5" title="Similarity Matching and Rele­vance Feedback"><subsection number="5.1" title="Cosine Similarity"><p>A standard Vector Space Model is used represent the query word and all the candidate words. The index is maintained in the form of a word-feature matrix, where each word image <i>w</i><i> </i>occupies one row of the matrix and all columns in a single row correspond to the moment values computed for the given word image.</p><p>When the user enters any query word, a lookup oper­ation is performed in the stored template to obtain the corresponding normalised word image for the input text. Feature extraction is performed on the word image to construct the query feature vector q. A cosine similarity score is computed for this query feature vector and all the rows of the word-feature matrix. The cosine similarity is calculated as follows:</p><doubt alpha="35.7" length="14" tooSmall="False" monospace="0.0">SIM(q,w)=-—(8)</doubt><doubt alpha="18.2" length="11" tooSmall="False" monospace="0.0">|q| * | W |</doubt><p>All the words of the document corpus are then ranked according to the cosine similarity score. The top choice returned by the ranking mechanism represents the word image which is most similar to the input query word.</p></subsection><subsection number="5.2" title="Relevance Feedback"><p>Since the word images present in the document corpus may be of poor print quality and may contain noise, the moment features computed may not be effective in rank­ing relevant word images higher in the obtained result. Also the presence of higher order moments may lead to inconsistency in the overall ranking of word images. To overcome this limitation, we have implemented a Rocchio's formula based manual Relevance Feedback mechanism. This mechanism re-formulates the query feature vector by adjusting the values of the individual moment orders present in the query vector. The relevance feedback mechanism assumes a user input after the presentation of the initial results. A user enters either a 1 denoting a result to be relevant or 0 denoting a result to be irrelevant. The new query vector is computed as follows:</p><p><i>a</i><i> </i><i>i=R </i><i>p</i><i> </i><i>j=NR</i> where <i>a</i><i> </i>, <i>p</i><i> </i>and 7 are term re-weighting constants. R de­notes a relevant result set and <i>NR </i>denotes a non-relevant result set. Forthis experiment, we chose <i>a </i>=1, <i>p = </i>0.75 and 7 = 0.25.</p><doubt alpha="61.1" length="36" tooSmall="False" monospace="0.0">qneu,=l-qoid +j^j-X^ "Tjvflf-Yjdi(9)</doubt></subsection></section><section number="6" title="Experiments and Results"><p>The moment based features seem more robust in handling different image transformations compared to commonly used feature sets for word spotting such as GSC features [1] and Gabor filter based features [2]. This can be ob-seerved in Figure 5. The first row of the image corre­sponds to different types of transformations applied to normal English handwritten word images ((a)) such as changing the image scale as in (b) or (c). The second row corresponds to linear ((f)) and scale transformation ((e)), when applied to the normal machine printed Hindi word image ((d)). Even after undergoing such transformations, the cosine similarity score between the moment features extracted from all image pairs is still close to 1 , which re­flects the strength of invariance of moment based features with respect to image transformations. Table 1 shows the cosine similarity score between all pairs of English word<page local="5" global="52"/></p><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">10</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.7</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">4</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">6</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">50</doubt><p>Commoo'oeal+h</p><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(a)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(b)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(c)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(d)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(e)</doubt><doubt alpha="33.3" length="3" tooSmall="False" monospace="0.0">(f)</doubt><p>Figure 5: Various forms of Image Transformations. (a) &amp; (d) Sample Word Image . (b),(c) &amp; (e) Scale Transforma­tion Examples (f) Linear Transformation Example .</p><table caption="Table 1: Cosine Similarity Score for English Transformed Word Image Pairs."></table><p>images. Table 2 shows the similarity score between all pairs ofhindi word images.</p><p>The data set for evaluating our methodology consists of documents in three scripts, namely English, Hindi and Sanskrit. For English, we used publicly available IAMdb [13] handwritten word images and word images extracted from George Washington's publicly available historical manuscripts [14]. The dataset for English consists of 707 word images. For Hindi, 763 word images were ex­tracted from publicly available Million Book Project doc­uments [12]. For Sanskrit, 693 word images were ex­tracted from 5 Sanskrit documents downloaded from the URL: http://sanskrit.gde.to/. For public testing and eval­uation, we have also made our dataset available at the lo­cation: http://cubs.buffalo.edu/ilt/dataset/.</p><p>For evaluating the system performance, we use the commonly used Average Precision Metric. Precision for</p><table caption="Table 2: Cosine Similarity Score for Hindi Transformed Word Image Pairs."></table><figure caption="Figure 6: Average Precision curve for English Word Spot­ting."></figure></section><section number="30" title="Queries"><figure caption="Figure 7: Average Precision curve for Hindi Word Spot­ting."></figure><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.9</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.8</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.6</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">5</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.5</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.4</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.2</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">2</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">7</doubt><doubt alpha="100.0" length="5" tooSmall="True" monospace="0.0">Query</doubt><doubt alpha="50.0" length="2" tooSmall="True" monospace="0.0">a.</doubt><doubt alpha="100.0" length="1" tooSmall="False" monospace="0.0">I</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.3</doubt><doubt alpha="0.0" length="1" tooSmall="True" monospace="0.0">0</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">20</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">40</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">60</doubt><table caption="Figure 6: Average Precision curve for English Word Spotting." class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Word Image Pair</b></p></td><td class="cell"><p><b>(d)</b></p></td><td class="cell"><p><b>(e)</b></p></td><td class="cell"><p><b>(f)</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>(d)</b></p></td><td class="cell"><p>1</p></td><td class="cell"><p>0.9662</p></td><td class="cell"><p>0.9312</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(e)</p></td><td class="cell"><p>0.9662</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0.9184</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(f)</p></td><td class="cell"><p>0.9312</p></td><td class="cell"><p>0.9184</p></td><td class="cell"><p>1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Word Image Pair</b></p></td><td class="cell"><p><b>(a)</b></p></td><td class="cell"><p><b>(b)</b></p></td><td class="cell"><p><b>(c)</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(a)</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0.9867</p></td><td class="cell"><p>0.9932</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(b)</p></td><td class="cell"><p>0.9867</p></td><td class="cell"><p>1</p></td><td class="cell"><p>0.9467</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>(c)</p></td><td class="cell"><p>0.9932</p></td><td class="cell"><p>0.9467</p></td><td class="cell"><p>1</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="6" global="53"/><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">10</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.7</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.9</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.8</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.6</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.5</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.4</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.2</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">0.3</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">20</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">40</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">60</doubt><table caption="Table 3: Average Precision rate for word spotting in all 3 Scripts"></table><doubt alpha="86.2" length="29" tooSmall="True" monospace="0.0">Avg Precision for few Queries</doubt></section><section number="50" title="Queries"><figure caption="Figure 8: Average Precision curve for Sanskrit Word Spotting."></figure><p>each query image was calculated at every recall level, and then averaged over to give an Average Precision per query. Figure 6 shows the average precision values for some query words in English. Figure 7 shows the average precision values for query words in Hindi. Figure 8 shows the average precision values for query words in Sanskrit.</p><p>The experimental results for all three scripts are sum­marised in Table 3. The Average Precision rates as shown in the table have been averaged over 30 queries in En­glish, 75 queries in Hindi and 100 queries in Sanskrit. As shown here, the system works better for machine printed text (71.18 and 87.88) as compared to handwritten (67.0). The best performance is seen with Sanskrit script (87.88), which has a variable length words allowing it to be more discriminative in its feature analysis as compared to other two scripts. Table 4 compares the performance of GSC based word spotting as reported in [1] against our method­ology. At 50% recall level, Moment based features per­form better than GSC based features for both handwritten English and machine printed Sanskrit documents.</p><p>We also evaluate the performance of Gabor Feature based word spotting method [2] on our dataset. Features are extracted using an array of Gabor filters having a scale from 4 pixels to 6 pixels and 8 orientations. Table 5 sum­marizes the performance of Gabor features based method as opposed to our Moment based system. As shown, Moment based features outperform Gabor based features in terms of average precision rates obtained for all 3 scripts used in the experiment.</p><table caption="Table 4: ComparisonofGSC andMoments basedfeatures at 50% recall level."></table></section><section number="7" title="Summary and Conclusion"><p>In this paper, we have proposed a framework for script in­dependent word spotting in document images. We have shown the effectiveness of using statistical Moment based features as opposed to some of the structural and profile based features which may constrain the approach to few scripts. Another advantage of using moment based fea­tures is that they are image scale and translation invariant which makes them suitable for font independent feature analysis. In order to deal with the noise sensitivity of the higher order moments, we use a manual relevance feed­back to improve the ranking of the relevant word images. We are currently working on extending our methodology to larger data sets and incorporating more scripts in future experiments.</p><table caption="Table 5: Comparison of Gabor filter based and Moments Features."></table><doubt alpha="100.0" length="2" tooSmall="False" monospace="0.0">CL</doubt><doubt alpha="100.0" length="3" tooSmall="True" monospace="0.0">Ave</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">30</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">70</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">80</doubt><doubt alpha="0.0" length="2" tooSmall="True" monospace="0.0">90</doubt><doubt alpha="0.0" length="3" tooSmall="True" monospace="0.0">100</doubt><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Script</b></p></td><td class="cell"><p><b>Before RF</b></p></td><td class="cell"><p><b>After RF</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>English</p></td><td class="cell"><p>66.30</p></td><td class="cell"><p>69.20</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Hindi</p></td><td class="cell"><p>71.18</p></td><td class="cell"><p>74.34</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sanskrit</p></td><td class="cell"><p>87.88</p></td><td class="cell"><p>92.33</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Script</b></p></td><td class="cell"><p><b>GSC</b></p></td><td class="cell"><p><b>Moments</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>English</p></td><td class="cell"><p>60.0</p></td><td class="cell"><p>71.6</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sanskrit</p></td><td class="cell"><p>90.0</p></td><td class="cell"><p>94.3</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><table class="main" frame="box" rules="all" border="1" regular="False"><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p><b>Script</b></p></td><td class="cell"><p><b>Gabor</b></p></td><td class="cell"><p><b>Moments</b></p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>English</p></td><td class="cell"><p>56.15</p></td><td class="cell"><p>66.30</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Hindi</p></td><td class="cell"><p>67.25</p></td><td class="cell"><p>71.18</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"><p>Sanskrit</p></td><td class="cell"><p>79.10</p></td><td class="cell"><p>87.88</p></td><td class="cell"></td></tr><tr class="row"><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td><td class="cell"></td></tr></table><page local="7" global="54"/></section><references><p>[1] S. N. Srihari, H. Srinivasan, C. Huang and S. Shetty, "Spotting Words in Latin, Devanagari and Arabic Scripts," Vivek: Indian Journal of Artificial Intelli­gence , 2006.</p><p>[2] Huaigu Cao, Venu Govindaraju, Template-Free Word Spotting in Low-Quality Manuscripts, the Sixth Inter­national Conference on Advances in Pattern Recogni­tion (ICAPR), Calcutta, India, 2007.</p><p>[3] S. Kuo and O. Agazzi, Keyword spotting in poorly printed documents using 2-d hidden markov models, in IEEE Trans. Pattern Analysis and Machine Intelli­gence, 16, pp. 842848,1994.</p><p>[4] M. Burl and P.Perona, Using hierarchical shape mod­els to spot keywords in cursive handwriting, in IEEE-CS Conference on Computer Vision and Pattern</p><p>Recognition, June 23-28, pp. 535540, 1998.</p><p>[5] A. Kolz, J. Alspector, M. Augusteijn, R. Carlson, and G. V. Popescu, A line oriented approach to word spot­ting in hand written documents, in Pattern Analysis and Applications, 2(3), pp. 153168, 2000.</p><p>[6] R. Manmatha and W. B. Croft, "Word spotting: In­dexing handwritten archives,"' InM. Maybury, editor, Intelligent Multimedia Information Retrieval Collec­tion, AAAI/MIT Press, Menlo Park, CA, 1997.</p><p>[7] T. Rath and R. Manmatha, .Features for word spot­ting in historical manuscripts,. in Proc. International Conference on Document Analysis and Recognition, pp. 218.222, 2003.</p><p>[8] T. Rath and R. Manmatha, .Word image matching us­ing dynamic time warping,. in Proceeding of the Con­ference on Computer Vision and Pattern Recognition (CVPR), pp. 521.527, 2003.</p><p>[9] Teh C.-H. and Chin R.T., "'On Image Analysis by the Methods of Moments,"' in IEEE Trans. Pattern Analysis and Machine Intelligence, 10, No. 4 , pp. 496513, 1988.</p><p>[10] Franz L. Alt, "'Digital Pattern Recognition by Mo­ments,"' in The Journal of the ACM , Vol. 9 , Issue 2 , pp. 240-258 , 1962.</p><p>[11] Jeff L. Decurtins and Edward C. Chen ,"' Keyword spotting via word shape recognition "' in Proc. SPIE Vol. 2422, p. 270-277, Document Recognition II, Luc M. Vincent; Henry S. Baird; Eds. , vol. 2422 , pp. 270-277, March 1995.</p><p>[12] Carnegie Mellon University - Million book project, URL: http://tera-3.ul.cs.cmu.edu/, 2007.</p><p>[13] IAM Database for Off-line Cursive Handwrit­ten Text, URL: http://www.iam.unibe.ch/ zim-merma/iamdb/iamdb.html.</p><p>[14] Word Image Dataset at CIIR - UMass , URL: http://ciir.cs.umass.edu/cgibin/downloads/downloads.cgi.</p></references></body></article>