5/10/2008

Improving Search on your web application

Searching is one of the common features supported by most of the applications, and if the amount of data is too high then many applications suffer the performance bottleneck in search response. Organizations are storing data into relational databases and large amount of transaction data is being stored for years. In cases where application wants to support advance search options it requires to join many relational tables internally and the query response becomes slower for more complex searches.

Advance searching can be improved with a different approach for implementing search on the application. A good option is to extract most of the relational data in a flat structure (non-normalized) form and create Lucene index for it. Performance of a flat structure lucence based index is much better compared to querying in the relational database directly. We have tested this for the data size of 100GB where it gives search results in milliseconds, as compared to minutes in case of relational database query.


A periodic update of Lucene index can be done to take care of new records, updates in the record. If your application has heavy updates then may be this would not be a suitable option for you, but if the application involves in-frequent updates then this would be the right thing to do.

No comments:

Post a Comment

Got something to say? Don't hold it! Tell it to us.

You Might Like

.....