First pypi release of repoze.pgtextindex for searching

Interested in faster/better searches in Pyramid/BFG without a change to your application, but don’t want a massive addition to your server software responsibilities?  Eager to keep transactional integrity?  repoze.pgtexindex uses the text indexing in PostgreSQL 8.4+ as a replacement for text indexing/searching in repoze.catalog’s zope.index.

Two Julys ago Shane Hathaway was working with us on the KARL project.  We had been discussing switching our KARL deployments to RelStorage.  The topic came up about text search, how important it is in our applications, and whether our search performance and quality were up to snuff.

We did some research on the text search capabilities in PostgreSQL.  Sure, it’s no Lucene or Xapian, but it looked like a significant improvement without a major change in deployment architecture.  Especially if someone was already looking at RelStorage and pgtextindex, they’d already have committed to supporting the dbms server.

That led to some experiments and we were pleased with the results.  But we’re really conservative about adding in software we can’t completely support so we put it on hold.  We then kept tinkering with it.  Chris Rossi got involved, Shane overhauled the transaction management, Chris added in implicit field weighting, etc.  We did another evaluation, then worked with Six Feet Up to make sure they were comfortable hosting/supporting RelStorage/pgtextindex/PostgreSQL.

And now we’re getting close to making the transition, so Chris worked with Shane and made a pypi release.

I’m excited about it.  Some might be attracted to big search solutions, but that’s quite a jump from the target for Zope-style applications.  I worked on a project that went big and it wasn’t exactly a pain-free drop-in.  If you’re a Pyramid/BFG person happy with zope.index then stick with it, that’s your least complex route.  But maybe you want much better search features, much faster search, half of your ZODB objects out of the ZODB and cache, and you’re already using PostgreSQL or comfortable with it.  If so, this is a much smaller step than adding a ginormous search engine and losing transactional integrity.

Some may prefer big search servers, some may prefer zope.index.  But I humbly submit that repoze.pgtextindex is another choice with certain positive qualities.


2 Responses to “First pypi release of repoze.pgtextindex for searching”

  1. Tweets that mention First pypi release of repoze.pgtextindex for searching « Chatterbox, Reloaded -- Says:

    […] This post was mentioned on Twitter by seantis and seantis, Planet Python. Planet Python said: Paul Everitt: First pypi release of repoze.pgtextindex for searching […]

  2. Max M Says:

    That sounds like a handy tool to have in the box.

    Especially in the context of pyramid. As pyramid is faster and built to scale well, having a fast and low resource search and indexing option is a good fit.

    Everything can be kept on a single, or a few, boxes for as long as possible.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: