28.12.05. A Glance at Zope 3.2 - Content Providers, Viewlets, and formlib

After ensuring some existing code would work under Zope 3.2 (still in beta testing at this point), I've been allowed this week to start work on a small internal (for now) project. I decided to take this opportunity to start looking at some new Zope 3.2 features. And while my experience with any of these is severely limited at this point, I want to mention them because they're all very interesting.

Content Providers, Viewlets, and Viewlet Managers

Two new top level packages in the Zope 3.2 distribution are zope.contentprovider and zope.viewlet. They're related, with the components in zope.viewlet building on zope.contentprovider.

What are these, and why are they important? In short, content providers are a simple construct that can be used to modularize web pages. Zope Page Templates have provided a decent way to modularize pages, using slots and macros to allow writing into multiple page parts more easily than a basic "insert header / body body body / insert footer" style template system. But it doesn't benefit from the flexibility that the component architecture can provide. As I've covered recently, A view in Zope is a multi-adapter, which discriminates against context (the object being viewed) and the request (HTTP request with skin information, FTP requests, and so on). A content provider adds to that, discriminating against the view as well. It's a smaller view used for that bigger view. More common terms for this are things like 'portlet'. There are some pages that you have, I'm sure, that you look at and you can see clear boxes that you are rendering in the common template, in included files, or being forced together a bit clumsily. You might be thinking to yourself "There's a lot of logic involved in rendering those navigation menus. I've got this great component architecture behind me, and I'm pulling that into a page with a 'var' and hoping it's there? Agh. And I don't want that navigation to show up like that in all of my shopping cart views... I need a different set of information over there during checkout..."

It appears to me that this is what content providers (and the viewlets built on top of them) are meant to solve. Based on JSR 168 - Portlet Specification, a content provider must implement only two methods: update() and render(). update() is the chance that the provider has to respond to the request and change state. It's always called first. Then render() is called to, um, render. On its own, it doesn't seem impressive. But its simplicity belies its power. zope.contentprovider doesn't provide an implementation, only some Interface specifications and a TALES namespace for using content provider components in Zope Page Template documents using the registered name. The component architecture will deliver the right component for the page being rendered. One has to look at zope.viewlet for base implementations.

zope.viewlet provides a common content provider in the form of a ViewletManager. The ViewletManager is probably the most common content provider one might call directly in a template. Viewlets are then registered with a ViewletManager, which will filter, sort, update, and render each Viewlet. A viewlet adds a forth discrimination - the viewlet manager. It all still builds on the component architecture - it's still multi-adapters underneath it all. So how might one use it? One example is the good old "side column" part of a page. With viewlets, one might register this as a viewlet manager with an interface name like ISideColumn or IAdditionalInformation. Then, one might provide a small viewlet for ISideColumn for all pages that shows the title, created date, modified date. Let's then say that you install a system that can find related content based on the tags or text of the current object. You could write a new viewlet that lists those pages in the ISideColumn - but only on pages that implement 'ITaggable'. The original template or viewlet manager don't have to be modified to render this, or to have any logic in them to discriminate against 'ITaggable'. The component architecture does the bulk of the work to find viewlets that can be drawn in a particular manager, and a particular viewlet manager implementation may do its own sorting and filtering of those viewlets - the base manager class provided in Zope 3.2, for example, filters out viewlets that the current user can't see.

What does this mean? What are the benefits of content providers / viewlets? I'm only speculating, based on very little personal experience so far, but I'm impressed with the simplicity of the interface. So here's a bulleted list of where I expect to benefit from this, and where I think others could too:

• Easier to specify page parts for current and future development. Build the page out of ViewletManagers, which are backed by Zope Interfaces, and then provide page parts against those. Navigation sections, promotional image displayers, menu items, footer links, sidebars, main areas. You can find these parts and formally specify them, and then provide the viewlets to flow into them.
• More intelligence in small parts. Viewlets may be simple or complex, but basically they can now have the same great (or small) intelligence that full view components have had. You know how it is - the middle left column of the page is expected to provide a search form, a navigation menu listing local pages, and a couple of small banner ads. Now all of those pieces can be more easily coded as separate components, but all in the same manner. This is possible in Zope 3.1, but content providers make this a lot easier.
• From a single code base, provide unique viewlets for the home page, section fronts, and common templates. This is another common little thing - lets say you have to have to add a hidden field to the little search form box used on every page, and the one on the front page needs a pull-down menu to allow discriminating results against different parts of the site. Normally, this little form is a small bit of code that's just managed in the templates. No big deal, there's only three - but they're all slightly different. You can't just copy and paste without paying attention to the unique requirements for each scenario. Putting that search form into a viewlet, you'd have access to the code used in each. Maybe you're using the same in each one. You can now make a specialized viewlet for the home page that adds the drop-down menu, while keeping it all tied together on the search page. You can also ensure that the 'search' sidebar widget does NOT draw on the search results page, or provide a more advanced search sidebar to filter results.
• Make a viewlet manager to handle HEAD links and easily add extra javascript library or CSS loading to pages that need them. I've commonly made page template 'slots' to allow me to load or add page-specific javascript or style sheets to individual templates. Using a viewlet manager for these, you can ensure that 'foolib.js' is there for every IFooListing based page.
• Use Page Templates, use another template language, render HTML directly - it doesn't matter. The interface is so simple, you no longer have to wonder about whether to use __call__ or __str__ or render as the best way to render a little component. 'update()' and 'render()' are all you need to worry about. Yet they can all plug into a larger page with ease, regardless of how that page is rendered. There are niceties for ZPT to make using content providers easy, but one could easily make a big "standard template" without a template language at all - just call and render a few principal ViewletManagers.
• Caching. There hasn't been a formal way of being able to cache fragments of pages. There's decent caching support (with more pluggable) in Zope 3, but no easy way of being able to apply them to parts of a page. With viewlets, it should be easier to put in caching. With more intelligence put into a viewlet, it could manage its own cached data or output. More effectively, one could put this in at the Viewlet Manager level without having to effect individual viewlets. The spec for content providers is simple enough that this could be plugged in a lot more easily and as needed.
• Based on simple standard. It's impressive to see that this is based on JSR 168. When I first heard this, some time ago, I flinched. JSR specifications are from a world that I just don't understand anymore, it seems. Yet this one is nice and simple and effective. It also should quiet some not-invented-here criticisms of Zope 3. At the same time, it has been specified and implemented in a way that fits with Zope 3. When I first looked at content providers and viewlets, I flinched then too. But as I started to use them, I realized that they didn't differ much from what I had already learned about views, widgets, and other multi-adapters. I've made 'views' with the express purpose of being rendered as part of a page, but they haven't followed a common interface. There are many other elements I see now that can fit into this system and ease use.

zope.formlib

The other addition I wanted to cover, briefly, is zope.formlib. Formlib is a general toolkit and API for doing intelligent web form development in Python. It's not a widget library - most of those are provided by the widgets in zope.app.form.browser, but could come from anywhere. It's not a validation framework - that's provided by zope.schema based fields (which work with Zope Interfaces). It does use both of those, and provides a lot of extra features to collect all of the input, apply the validation or collect errors, do invariant based validation ('end date must be later than start date'), dispatch to different responsive actions, and more.

It's a toolkit capable of building powerful base classes and individual uses for forms that are simple (just a few fields) or very complex. There were some form views provided in Zope 3.0 and 3.1 that do similar, but they weren't so easy to use and understand and didn't handle complex forms. Custom field and widget combinations required a lot of custom work, and so did dispatching to multiple actions based on success, failure, and so on. Lastly, the older (non-formlib) techniques seemed to prefer being built via the Zope 3 component configuration language when it really required Python to get the kind of flexibility that form development requires.

In wrap-up, formlib is promising because it shows a preference for using Python and some of the recent features such as decorators over giving everything over to configuration based automation. Content Providers are a simple specification for modularizing web pages into manageable components, with viewlets and viewlet managers providing a reasonable base implementation to work with. The dual layers provide a boundary between a basic spec and small set of tools (the content provider interface, a TALES expression for use in zope page templates) that have broad usage and more specialized providers, built from real-world usage and previous art and examples like JSR 168.