Pyramid Training: Gun for Hire

tl;dr I’d love to go do some Pyramid training in the coming months. Particularly if there’s a chance to speechify.

For the San Fran Plone Conf, I put together a “Pyramid for Humans” training course. I really enjoyed it. I tried to create the kind of training course I’d want to attend: very hands-on, one step at a time, source code available, broad-but-shallow, with a deep emphasis on hilarity. Chris McDonough sat in on it to make sure I didn’t poke myself in the eye on anything.

The course received very good reviews, prompting me to…do nothing. I stupidly didn’t go conduct it again. I have submitted a condensed version as a tutorial for PyCon, but then it occurred to me: “Why wait?”

Got a small pile of beginners in your organization that you’d like to get up to speed on Python web development? Got a larger group at a conference or user group that might want a hands-on tour of Pyramid? Interested in a speech covering 19 years of mistakes in web development and 15 years of bad open source decisions?

As a trainer, I’m not: (a) cheap, (b) expensive, or (c) boring. I’m up for doing customized training if you have some specific topics.

Hackberry, Linux, Python, Minecraft

My son wanted to run a Minecraft Server. I had a old, wheezing, 2006 Macbook, so I set up a Minecraft server for him and his friends.

Then I decided to do a little adventure.

The Macbook wasn’t going to live long, and I knew I wanted him to do some more learning on computing. So we went the SoC route and got the 1Gb Hackberry for $75 or so bucks. Dual core 1.2 Ghz ARM with 1 GB RAM, HDMI, 2 USB, Wifi, Bluetooth, and an SD reader.

These things come with onboard Android so you can just plug them in, hook up the HDMI and keyboard/mouse, and all works. And it did. It was very unnerving, the paucity of actual learning required. My son made a case out of Legos, I hooked up the Wifi, and the Google Play store actually had an Android version of the Minecraft Server.

Too easy. Fortunately the Android “server” was fatally flawed. It stopped if you did anything else in Android. So we flashed Linaro Desktop onto an SD card and went down the rabbit hole.

And it worked. The first time. I just had to fix the wireless networking. After that, a flurry of apt-gets later, and I had a Minecraft server running and Python installed. 10k Pystones makes it 1/4th the speed of my 2006 Macbook. But hey, it’s in a case made out of Legos!

Humans, Python, web development, tutorial next week

(Aimed at Python folks.)

If you’re interested in learning Python web development using Pyramid, and you’re not a framework guru, I’m giving a “Pyramid for Humans” tutorial next week in San Francisco. I’m aiming this particularly at web developers who are responsible for the UI/UX and customer interactions, versus people who like talking about thread locals.

I’ve really enjoyed working on the material for this and I’m looking forward to going through it next week.

Pyramid tutorial at Plone Conf

Let’s all pretend that this post went out 2 months ago, which would have been a grown-up and mature time to have promoted it.

In a couple of weeks I’m giving a Pyramid tutorial at the Plone Conference in San Francisco. I’m hardly an expert, so this is pitched as “Pyramid for Humans.” I’ll start at the very beginning, establish a UI-first approach to building an app, then gradually add much of the machinery needed for a project.

I’d really like it if you attended. [wink]

Joking aside, I’m really hoping I can get a crowd of first-starters there. Women and men who aren’t framework gods, who like using tools more than making them, and we build a nice base understanding of getting started in Pyramid. I want this tutorial to kick ass.

Drop me a line if you have any questions or suggestions about what should go in “Pyramid for Humans.”

Giving a “Pyramid for Humans” tutorial at Plone Conf, Nov 1-2

Yep, me. Paulie. Giving a tutorial on Pyramid at the conference. Stop snickering!

Ok, sure, Dad (Chris McDonough) will be there supervising. But I’m hoping to give a whirlwind tutorial of Pyramid stuff for people like me. Civilians. It won’t be too in-depth, and it will for damn straight make sure you are successful at accomplishing each thing we cover.

The registration page has more details on who it is aimed and not aimed at, as well as the topics I hope to cover. I’ve started writing some of the material and reading some previous tutorial material from Chris and Carlos de la Guardia.

If you’re looking for a gentle but productive introduction to Python web development using Pyramid, come join us. I’m really excited about this and expect it to be a crap-ton o’ fun.

Giving a keynote at PyCon DE

Filed under “they must REALLY be scraping the bottom of the barrel to dig up ME“, I’ve been asked to give a keynote at the PyCon DE conference in Leipzig. I’m 15:30 to 16:30 on Thursday, October 6. Lightning talks are just after me, so I’m not completely the thing in between the audience and happy hour.

I used to give talks all…the…time. And now I don’t. Haven’t in what seems like a long time, unless giving a halftime talk to a team of girls lacrosse players qualifies. I’ve been out of the game for a long time and the relevance of my perspective is a bit rusty.

So I’m going to stick to perspective and look backwards. I’m collecting a bunch of moments over the last 18 years participating in steering Python/Zope/Plone/Pyramid projects. Hopefully funny, possibly poignant, certainly embarrassing. I also hope to include some anecdotes and wisdom from people I’ve met along the way who have guided projects over a long period of time.

I always used to say my talks had a content-free guarantee: if you actually learned something, I’d give you your money back. Definitely doubling down on that for this talk.

Kinda liking PyCharm

I’ve been tinkering around with IDEs for a while.  It’s hard to justify it…every time I try, I spend a tremendous amount of time using the hoped-for new tool, using it for real, only to eventually find the fatal flaws.  Then it’s back to Emacs with a dozen flowers, a box of chocolates, and the hangdog expression that says “Sorry I set my eyes on another.”

This has happened for Pydev/Eclipse, Aptana, Textmate, and Komodo.  Each brought down by some level of (a) horrible performance on OS X, (b) extreme amount of tinkering to get something working such as indentation, or (c) a realization that the cost (Komodo) meant I’d be the only one I know using.

Thought I’d give PyCharm a try.  It’s based on a larger IDE project so it automatically gains a lot of features, such as really good HTML and Javascript support.  Getting it working with virtualenv and buildout wasn’t excruciating, in fact, the added specific support for it.  And the performance was pretty good.  Considering I still can’t get decent Javascript indentation, even after manual help from others, it was a good sign.

Last week my trial run ran out.  I could have chintzed by asking them for an open source license.  But I plopped down the modest cost, expecting that I would find the fatal flaw about 10 minutes after the credit card charge cleared.

But today I looked at my screen and realized…damn, I’m pretty happy.  Still scratching the surface.  When they add reST/Sphinx support later this year, I’ll be even happier.  Maybe happy enough to put out a bounty for Chameleon/Pyramid support.

“Did you mean FOOBAR?” in PostgreSQL text indexing

This is a reminder-to-self kind of post.  Chris Rossi and I were talking about some of the “improved search” features we have planned for KARL thanks to pgtextindex and the use of PostgreSQL’s under-appreciated text indexing/retrieval story.

One thing we think we could add is “Did you mean?” support.  That is, if someone did a typo on a search term, offer some alternatives.  It’s a valuable feature (at least to me.)

The most natural thing to think about is brute-force spellchecking.  That has a number of flaws.  First, lots of things (names, for example) won’t be in any dictionary, much less some default language dictionary.  Second, you can’t show a bunch of corrections, which ones should you show?  Finally, what if you show a correction for which the word doesn’t occur in the corpus?

In Fuzzy String Matching With Trigram and Trigraphs you see how PostgreSQL helper functions can, well, help.  You can compare a term to all the reduced forms of all the words indexed in all the documents.  Then issue a query that does a soundex-style comparison to the query terms.  All using optimized indices, weighting, and scoring.  Perhaps you could also use corpus statistics to narrow the list down to words that occur in many documents.

Chris and I previously came across this when he had a neat idea on speeding up prefix searches.  Rather than do a prefix, instead find all the expansions of words/lexemes for that prefix which occur in the corpus, select the 100 most likely candidates, and do full-word searches for those 100 words.  You get better performance by omitting prefixes.  One might assert you get better quality results by letting all the full-word scoring machinery, synonyms, stemming, etc. kick in.

It’s interesting to look at some of the machinery PostgreSQL has which doesn’t get talked about much.  Like the ltree for hierarchies and tree structures.

First pypi release of repoze.pgtextindex for searching

Interested in faster/better searches in Pyramid/BFG without a change to your application, but don’t want a massive addition to your server software responsibilities?  Eager to keep transactional integrity?  repoze.pgtexindex uses the text indexing in PostgreSQL 8.4+ as a replacement for text indexing/searching in repoze.catalog’s zope.index.

Two Julys ago Shane Hathaway was working with us on the KARL project.  We had been discussing switching our KARL deployments to RelStorage.  The topic came up about text search, how important it is in our applications, and whether our search performance and quality were up to snuff.

We did some research on the text search capabilities in PostgreSQL.  Sure, it’s no Lucene or Xapian, but it looked like a significant improvement without a major change in deployment architecture.  Especially if someone was already looking at RelStorage and pgtextindex, they’d already have committed to supporting the dbms server.

That led to some experiments and we were pleased with the results.  But we’re really conservative about adding in software we can’t completely support so we put it on hold.  We then kept tinkering with it.  Chris Rossi got involved, Shane overhauled the transaction management, Chris added in implicit field weighting, etc.  We did another evaluation, then worked with Six Feet Up to make sure they were comfortable hosting/supporting RelStorage/pgtextindex/PostgreSQL.

And now we’re getting close to making the transition, so Chris worked with Shane and made a pypi release.

I’m excited about it.  Some might be attracted to big search solutions, but that’s quite a jump from the target for Zope-style applications.  I worked on a project that went big and it wasn’t exactly a pain-free drop-in.  If you’re a Pyramid/BFG person happy with zope.index then stick with it, that’s your least complex route.  But maybe you want much better search features, much faster search, half of your ZODB objects out of the ZODB and cache, and you’re already using PostgreSQL or comfortable with it.  If so, this is a much smaller step than adding a ginormous search engine and losing transactional integrity.

Some may prefer big search servers, some may prefer zope.index.  But I humbly submit that repoze.pgtextindex is another choice with certain positive qualities.

Chris McDonough on pluggable apps

We’re from the world of Zope and Plone, so we have a long history on the topic of frameworks and pluggability.  We’ve helped foist the good, the bad, and the ugly on large populations of developers over the years.  So we’re continuing to learn the right balance on frameworkiness.

Chris wrote a very good email yesterday on the Pylons mailing list on the topic of Pyramid’s pluggability story.  The “Pylons Project” wants to have a story over the long haul.  But we’re still in the mode of thinking, learning, listening, and then building.

To summarize Chris’s note (at least from my POV):

• Pyramid provides a pretty attractive extensibility story for application developers.
• Pluggability is harder, because a plug-in will need cooperating systems (storage, authentication, etc.) and Pyramid isn’t going to have opinions on all that.
• Your application will have opinions, though, thus you can provide plug points built with Pyramid’s extensibility story.
• A plugin-story’s success is directly related to the ruthlessness of the decisions and opinions being made.  The more you define the surface area by making the choices, the more viable the plug point.  Over-generalization is the enemy.
• The Pylons Project would like to make an application on Pyramid that has opinions, provides higher-level services, and thus has a strong plugin story.

At the end of the day, the Python world has a lot of great alternatives already for web frameworks.  Anything new that tries to emerge needs fresh ideas.  I feel like extensibility and pluggability are still areas ripe for new thoughts.

While many might want less (e.g. single file microframeworks) but there might be a constituency for those that want more.