If you’re interested in learning Python web development using Pyramid, and you’re not a framework guru, I’m giving a “Pyramid for Humans” tutorial next week in San Francisco. I’m aiming this particularly at web developers who are responsible for the UI/UX and customer interactions, versus people who like talking about thread locals.

I’ve really enjoyed working on the material for this and I’m looking forward to going through it next week.

In a couple of weeks I’m giving a Pyramid tutorial at the Plone Conference in San Francisco. I’m hardly an expert, so this is pitched as “Pyramid for Humans.” I’ll start at the very beginning, establish a UI-first approach to building an app, then gradually add much of the machinery needed for a project.

I’d really like it if you attended. [wink]

Joking aside, I’m really hoping I can get a crowd of first-starters there. Women and men who aren’t framework gods, who like using tools more than making them, and we build a nice base understanding of getting started in Pyramid. I want this tutorial to kick ass.

Drop me a line if you have any questions or suggestions about what should go in “Pyramid for Humans.”

Ok, sure, Dad (Chris McDonough) will be there supervising. But I’m hoping to give a whirlwind tutorial of Pyramid stuff for people like me. Civilians. It won’t be too in-depth, and it will for damn straight make sure you are successful at accomplishing each thing we cover.

The registration page has more details on who it is aimed and not aimed at, as well as the topics I hope to cover. I’ve started writing some of the material and reading some previous tutorial material from Chris and Carlos de la Guardia.

If you’re looking for a gentle but productive introduction to Python web development using Pyramid, come join us. I’m really excited about this and expect it to be a crap-ton o’ fun.

I used to give talks all…the…time. And now I don’t. Haven’t in what seems like a long time, unless giving a halftime talk to a team of girls lacrosse players qualifies. I’ve been out of the game for a long time and the relevance of my perspective is a bit rusty.

So I’m going to stick to perspective and look backwards. I’m collecting a bunch of moments over the last 18 years participating in steering Python/Zope/Plone/Pyramid projects. Hopefully funny, possibly poignant, certainly embarrassing. I also hope to include some anecdotes and wisdom from people I’ve met along the way who have guided projects over a long period of time.

I always used to say my talks had a content-free guarantee: if you actually learned something, I’d give you your money back. Definitely doubling down on that for this talk.

This has happened for Pydev/Eclipse, Aptana, Textmate, and Komodo.  Each brought down by some level of (a) horrible performance on OS X, (b) extreme amount of tinkering to get something working such as indentation, or (c) a realization that the cost (Komodo) meant I’d be the only one I know using.

Thought I’d give PyCharm a try.  It’s based on a larger IDE project so it automatically gains a lot of features, such as really good HTML and Javascript support.  Getting it working with virtualenv and buildout wasn’t excruciating, in fact, the added specific support for it.  And the performance was pretty good.  Considering I still can’t get decent Javascript indentation, even after manual help from others, it was a good sign.

Last week my trial run ran out.  I could have chintzed by asking them for an open source license.  But I plopped down the modest cost, expecting that I would find the fatal flaw about 10 minutes after the credit card charge cleared.

But today I looked at my screen and realized…damn, I’m pretty happy.  Still scratching the surface.  When they add reST/Sphinx support later this year, I’ll be even happier.  Maybe happy enough to put out a bounty for Chameleon/Pyramid support.

One thing we think we could add is “Did you mean?” support.  That is, if someone did a typo on a search term, offer some alternatives.  It’s a valuable feature (at least to me.)

The most natural thing to think about is brute-force spellchecking.  That has a number of flaws.  First, lots of things (names, for example) won’t be in any dictionary, much less some default language dictionary.  Second, you can’t show a bunch of corrections, which ones should you show?  Finally, what if you show a correction for which the word doesn’t occur in the corpus?

In Fuzzy String Matching With Trigram and Trigraphs you see how PostgreSQL helper functions can, well, help.  You can compare a term to all the reduced forms of all the words indexed in all the documents.  Then issue a query that does a soundex-style comparison to the query terms.  All using optimized indices, weighting, and scoring.  Perhaps you could also use corpus statistics to narrow the list down to words that occur in many documents.

Chris and I previously came across this when he had a neat idea on speeding up prefix searches.  Rather than do a prefix, instead find all the expansions of words/lexemes for that prefix which occur in the corpus, select the 100 most likely candidates, and do full-word searches for those 100 words.  You get better performance by omitting prefixes.  One might assert you get better quality results by letting all the full-word scoring machinery, synonyms, stemming, etc. kick in.

It’s interesting to look at some of the machinery PostgreSQL has which doesn’t get talked about much.  Like the ltree for hierarchies and tree structures.

Two Julys ago Shane Hathaway was working with us on the KARL project.  We had been discussing switching our KARL deployments to RelStorage.  The topic came up about text search, how important it is in our applications, and whether our search performance and quality were up to snuff.

We did some research on the text search capabilities in PostgreSQL.  Sure, it’s no Lucene or Xapian, but it looked like a significant improvement without a major change in deployment architecture.  Especially if someone was already looking at RelStorage and pgtextindex, they’d already have committed to supporting the dbms server.

That led to some experiments and we were pleased with the results.  But we’re really conservative about adding in software we can’t completely support so we put it on hold.  We then kept tinkering with it.  Chris Rossi got involved, Shane overhauled the transaction management, Chris added in implicit field weighting, etc.  We did another evaluation, then worked with Six Feet Up to make sure they were comfortable hosting/supporting RelStorage/pgtextindex/PostgreSQL.

And now we’re getting close to making the transition, so Chris worked with Shane and made a pypi release.

I’m excited about it.  Some might be attracted to big search solutions, but that’s quite a jump from the target for Zope-style applications.  I worked on a project that went big and it wasn’t exactly a pain-free drop-in.  If you’re a Pyramid/BFG person happy with zope.index then stick with it, that’s your least complex route.  But maybe you want much better search features, much faster search, half of your ZODB objects out of the ZODB and cache, and you’re already using PostgreSQL or comfortable with it.  If so, this is a much smaller step than adding a ginormous search engine and losing transactional integrity.

Some may prefer big search servers, some may prefer zope.index.  But I humbly submit that repoze.pgtextindex is another choice with certain positive qualities.

Chris wrote a very good email yesterday on the Pylons mailing list on the topic of Pyramid’s pluggability story.  The “Pylons Project” wants to have a story over the long haul.  But we’re still in the mode of thinking, learning, listening, and then building.

To summarize Chris’s note (at least from my POV):

• Pyramid provides a pretty attractive extensibility story for application developers.
• Pluggability is harder, because a plug-in will need cooperating systems (storage, authentication, etc.) and Pyramid isn’t going to have opinions on all that.
• Your application will have opinions, though, thus you can provide plug points built with Pyramid’s extensibility story.
• A plugin-story’s success is directly related to the ruthlessness of the decisions and opinions being made.  The more you define the surface area by making the choices, the more viable the plug point.  Over-generalization is the enemy.
• The Pylons Project would like to make an application on Pyramid that has opinions, provides higher-level services, and thus has a strong plugin story.

At the end of the day, the Python world has a lot of great alternatives already for web frameworks.  Anything new that tries to emerge needs fresh ideas.  I feel like extensibility and pluggability are still areas ripe for new thoughts.

While many might want less (e.g. single file microframeworks) but there might be a constituency for those that want more.

Alex Marandon wrote exactly the article I wished I would have written.  He doesn’t presume you know Zope.  He then takes the (from my POV, super-easy) ideas of traversal and persistence, and introduces them in a simple, obvious way.

Thus I’ve watched with some amount of curiosity an anti-jQuery-UI meme emerging in some projects I’ve participated in.  It isn’t just that they favor jQuery Tools.  Rather, they also seem intent on some broad-brushed jQuery UI bashing to go with it.  Most of the time the word “bloat” emerges, and I thought I’d look into it.

“Bloat” is usually something that can be easily measured.  For example, ahem, some open source content management systems ship with a significant amount of code, thus a lot of “bloat”.  But when looking at jQuery UI (jqUI) vs. jQuery Tools (jqT), I found:

• jqUI: 19 Kb
• jqT: 2.4 Kb

That’s the size of the JS for each, gzip’d (in the way any normal production webserver would deliver it.)  For the comparison, I made a jQuery UI download that matches the same widget framework features used by Plone, to make it even.  It isn’t utterly apples-to-apples: jQuery UI also has CSS and images which add marginally to the Kb needed on a site.

But just to put this into perspective, the difference is about 1/2 the size of the Plone logo.  I don’t think “bloat” could apply to pageload impact, but even if it did…who cares?  Any intelligent site uses far-future expires and versioned URLs to ensure visitors only need one request for the JS library…ever.

So perhaps “heavy” and “bloat” don’t mean measurable things.  Perhaps the point is conceptual heaviness.  It’s true that the jQuery UI widget framework adds some ceremony in the constructor and method dispatch.  But two notes in its defense.

First, the method dispatch pattern is the one recommended by the jQuery core team for plugin extensibility.  Well, I’m sure “jQuery core team” probably includes lots of viewpoints, but the page I read on the topic seemed authoritative.

Second, there is a reason the ceremony is there: widget re-use and extensibility.  There’s always a price to be paid for this.  But when building a big system, the price can be worthwhile.  I suspect systems like Plone will be paying that price at some point, just with perhaps different ceremony.

I’ve seen “bloat” connected to “CSS bloat”.  I’m also not sure what specifically that means, I haven’t seen any comparisons to jqT selector hierarchies.  I certainly can sympathize with this point: the jQuery UI CSS Framework has lots of classes for things such as .ui-widget and .ui-widget-container and .ui-widget-content.  But so does Plone’s templating, and for the same reason as the above: large systems need fine-grained extensibility and customizability.

Personally I’m happy jqUI went that way, as they are busy stripping their main widgets down dramatically, composed from a small set of building blocks that get extended.  Even better, they documented their selector hierarchy in a fairly-visible contract.

At the end of the day, my best guess is that “bloat” is being used in a non-measurable way as a style choice.  Which is absolutely fine.  It’s great to say “I prefer jQuery Tools”.  It’s great that both exist, as I think they target different audiences.  jQuery Tools when you want to quickly get something done without learning a lot of ceremony, jQuery UI when you have a big system that needs a focus on extensibility and customization.

Maybe I have it wrong, and “bloat” is something specific and empirical.  Either way, they are both good choices and fairly similar in features, size, and support.