At the end of May we made the KARL cutover from KARL2 to KARL3. We have now had almost 8 weeks in production, so we can form impressions about some of the decisions.
For example, we’re using modwsgi as a WSGI server. In a way, this was a surprising decision. Chris and I were both a bit skeptical about whether it was a risk worth taking, as we hadn’t used it “in anger” beyond the repoze.org site. As somewhat a throwaway test, I set it up on the test site we used for OSI to do the user acceptance testing for the 15 or so milestone deliverables on KARL, but with the idea that it wasn’t a permanent decision.
By the time we setup the deployment server, we had months of living with modwsgi under our belt, somewhat by accident. Far more important, we added Shane Hathaway to the project. Like Chris Rossi, Shane has been a boon for KARL in many ways. In this case, Shane had experience with modwsgi and said he would make sure we could stand behind it.
So we put it into production. We’re running on an 8-core box setup as a Xen server, with the OSI instance getting 3 CPUs. We started with modwsgi setup to run 2 BFG application processes (and thus ZEO clients), each with 2 threads. modwsgi gave us some simplification: we didn’t need Apache/mod_proxy + BFG/Paster, with the latter managed by supervisor, possibly with a load balancer in between. Instead, we let modwsgi in daemon mode handle that.
We also found that we could upgrade the software on the server, send a -GRACEFUL restart to Apache, and have a zero-downtime update to the server. Which was nice.
Later, though, we found a really useful benefit to having modwsgi in the equation. We then found that our LiveSearch implementation (and another part of the application) were slower (up to a second for a request) than other parts. So we put in an Apache alias that matched on those URL types and sent them to a separately-configured BFG instance we call the “ghetto”. This instance is primarily for catalog requests, so we changed the ZEO cache to be *very* high, but we also flush from the cache on each request any object that isn’t in the catalog. This has been a boon.
I have an interest in looking later at some more options we gain from modwsgi. For example, file delivery using wsgi.file_wrapper (which modwsgi fixed its implementation bugs in later releases.) Also, modwsgi has some facilities for setting limits on the life of a request.
Still, regarding just the basics, we haven’t had much of a hitch at all. So far, so good.