Interested in faster/better searches in Pyramid/BFG without a change to your application, but don’t want a massive addition to your server software responsibilities? Eager to keep transactional integrity? repoze.pgtexindex uses the text indexing in PostgreSQL 8.4+ as a replacement for text indexing/searching in repoze.catalog’s zope.index.
Two Julys ago Shane Hathaway was working with us on the KARL project. We had been discussing switching our KARL deployments to RelStorage. The topic came up about text search, how important it is in our applications, and whether our search performance and quality were up to snuff.
We did some research on the text search capabilities in PostgreSQL. Sure, it’s no Lucene or Xapian, but it looked like a significant improvement without a major change in deployment architecture. Especially if someone was already looking at RelStorage and pgtextindex, they’d already have committed to supporting the dbms server.
That led to some experiments and we were pleased with the results. But we’re really conservative about adding in software we can’t completely support so we put it on hold. We then kept tinkering with it. Chris Rossi got involved, Shane overhauled the transaction management, Chris added in implicit field weighting, etc. We did another evaluation, then worked with Six Feet Up to make sure they were comfortable hosting/supporting RelStorage/pgtextindex/PostgreSQL.
And now we’re getting close to making the transition, so Chris worked with Shane and made a pypi release.
I’m excited about it. Some might be attracted to big search solutions, but that’s quite a jump from the target for Zope-style applications. I worked on a project that went big and it wasn’t exactly a pain-free drop-in. If you’re a Pyramid/BFG person happy with zope.index then stick with it, that’s your least complex route. But maybe you want much better search features, much faster search, half of your ZODB objects out of the ZODB and cache, and you’re already using PostgreSQL or comfortable with it. If so, this is a much smaller step than adding a ginormous search engine and losing transactional integrity.
Some may prefer big search servers, some may prefer zope.index. But I humbly submit that repoze.pgtextindex is another choice with certain positive qualities.