David Mertz (mertz@gnosis.cx), Programmer, Gnosis Software, Inc.
Summary: In the previous installment of this series, David looked at some higher-level techniques for writing Web services, including serving dynamic pages using the .rpy extension. In this article, he moves on to look at dynamic Web serving, and how to generate dynamic Web pages using the Woven application for templating pages.
Interacting with a Web browser
In Part 2 of this series, I looked at dynamic Web pages served by Twisted using the .rpy extension. But those initial versions of a weblog server were only minimally dynamic. I used HTML tags to force a page to refresh periodically, and upon each refresh performed a bit of calculation to determine the relevant recent hits. But there was no user configuration aspect to the server.
The first thing we will look at in this installment, therefore, is how to configure user interaction into the same basic dynamic page framework we looked at before. But before I start, let me include a quick reminder of how to launch a Twisted Web server for readers who may not have read the prior installments of this series.
Creating a "pickled application" is usually the best approach, and it can be done purely with command-line options. You do nothave to do it this way. If you like, you are free to include some extra capabilities inside the basic Web server (such as maintaining persistent data across users and sessions), but it is not necessary to write any custom code. Creating the pickled application looks something like this:
mktap web --path ~/twisted/www --port 8080Launching it consists of this:
twistd -f web.tapThat's it. Any HTML or .rpy files that happen to be in the ~/twisted/www base directory (or subdirectories) will be served to clients on port 8080. Actually, you can serve whatever file type you like, but the .rpy files will be treated as special dynamic scripts.
The dynamic page config_refresher.rpy is a bit longer than any presented in the prior installment; it includes HTML templates in its body rather than importing them. Let's first look at the setup code:
Listing 1. Dynamic script config_refresher.py (setup)
from twisted.web import resource, server
from persist import Records
from webloglib import log_fields, COLOR
from urllib import unquote_plus as uqp
fieldnames = """ip timestamp request status
bytes referrer agent""".split()
field_dict = dict(zip(fieldnames, range(len(fieldnames))))
Other than a few imports that we have seen in prior installments, I map field names to their positions in the tuple returned bylog_fields(). Do notice also the use of a custom persist module that will hold the weblog in memory within the Twisted Web server, so that the whole log file does not need to be read each time any client requests some records. Next, the HTML templates:
Listing 2. config_refresher.py script (templates)
TOP = '''<html><head><title>Weblog Refresher</title>
<META HTTP-EQUIV="Refresh" CONTENT="30"/></head>
<body>
<table border="1" width="100%%">
<tr bgcolor="yellow">
<form action="http://gnosis.cx:8080/config_refresher.rpy"
method="GET">
<td> IP <input type="checkbox" name="ip" %s/> </td>
<td> Timestamp <input type="checkbox" name="timestamp" %s/></td>
<td> Request <input type="checkbox" name="request" %s/></td>
<td> Status <input type="checkbox" name="status" %s/></td>
<td> Bytes <input type="checkbox" name="bytes" %s/></td>
<td> Referrer <input type="checkbox" name="referrer" %s/></td>
<td> Agent <input type="checkbox" name="agent" %s/></td>
<td> <input type="submit" value="Change Fields"></td>
</form>
</td></tr>
<table border="0" cellspacing="0" width="100%%">'''
ROW = '<tr bgcolor=" %s">%s</tr>\n'
END = '</table></body></html>'
COLOR = ['white','lightgray']
END = '''</table></body></html>'''
Setting up an HTML form is not too mysterious, but one little trick in the example is to interpolate the string "checked" into those checkboxes in the HTML that have been checked.
Listing 3. config_refresher.py script (persistence)
records = registry.getComponent(Records)
if not records:
records = Records()
registry.setComponent(Records, records)
The Twisted registry works as described in the previous installment. It is simply the place where the latest records in the Web log file are held. Finally, we create a Resource, with a corresponding .render() method -- this does the actual page creation:
Listing 4. config_refresher.py script (rendering)
class Resource(resource.Resource):
def render(self, request):
showlist = []
for field in request.args.keys():
showlist.append(field_dict[field])
showlist.sort()
checked = [""] * len(fieldnames)
for n in showlist:
checked[n] = 'checked'
request.write(TOP % tuple(checked))
odd = 0
for rec in records.getNew():
hit = [field.strip('"') for field in log_fields(rec)]
flds='\n'.join(['<td>%s</td>'%hit[n] for n in showlist])
request.write(ROW % (COLOR[odd],
uqp(flds).replace('&&',' &')))
odd = not odd
request.write(END)
request.finish()
return server.NOT_DONE_YET
resource = Resource()
The main new thing in this Resource is the access to the request.args attribute. Generally, this attribute is similar to theFieldStorage class in the cgi module -- it collects any information passed with the page request, both GET and PUT data. Twisted's request data is a dictionary of passed values; in our case we are just interested in which checkboxes' fields are passed in and which are not. But it would follow the same pattern if we wanted to check some values stored in request.args. You might, for example, add options to filter based on field values (and choose this with a text entry or an HTML listbox).
Templating with Woven
The dynamic pages we have looked at so far have all been conceptually similar to a CGI approach. Twisted asynchronous server is faster -- and it especially saves time to avoid the overhead of opening a new process for each script request. But fastcgi ormod_python achieve a similar speedup. There is nothing all that special about Twisted in this regard.
One way to move Web application development to a higher level is to use Woven. In concept, Woven is somewhat similar to PHP, ASP (Active Server Pages), or JSP (JavaServer Pages). That is, Woven XHTML pages are not simply the pages delivered to browsers, but rather templates or skeletons of pages that are filled in programmatically. However, the separation between code and HTML is a bit different in Woven than in those page-embedding technologies. You do not write Python code directly inside a Woven template. Instead you define a series of custom XHTML attributes on normal tags that let external code enhance and manipulate the page in preparation for delivery to the browser client.
The model attribute determines the data that is used for expanding the XHTML element it is attached to. The idea is that a Model represents the "business logic" of an application -- how the data content of a page is determined. The view attribute, in contrast, determines the particular presentation of the generated data. There is also the concept of a Controller in Woven, which is the code that combines the Model with the View of a node (in other words, an XHTML element). This last part is usually handled by aPage object, which is a class that can be specialized.
The nomenclature behind Woven is admittedly a bit difficult, and unfortunately, the HOWTO documents at the Twisted Matrix Web site do almost as much to obscure matters as they do to illuminate them. It is hard going to figure out exactly how to use Woven. I do not claim to wholly understand Woven concepts myself, but Twisted user Alex Levy (please see Resources for a link to his page) was kind enough to help me through developing the example I present below. Still, there is quite a bit you can do with Woven, so it is worth working through.
The first step for a Woven application is to develop a template file (or files). These are simply XHTML files with special attributes, for example:
Listing 5. WeblogViewer.xhtml template
<html>
<head>
<title>Weblog Viewer</title>
<meta HTTP-EQUIV="Refresh" CONTENT="30" />
<style type="text/css"><!--
div.info {
background-color: lightblue;
padding: 2px dotted; }
table th, table td {
text-align: left;
cellspacing: 0px;
cellpadding: 0px; }
table.log {
border: 0px;
width: 100%; }
table.log tr.even { background-color: white; }
table.log tr.odd { background-color: lightgray; }
--></style>
</head>
<body>
<div class="info">
You are displaying the contents of
<code model="filename" view="Text">filename</code>.
</div>
<table border="0" cellspacing="0" width="100%"
class="log" model="entries" view="List">
<tr bgcolor="yellow" pattern="listHeader">
<th>Referrer</th><th/>
<th>Resource</th>
</tr>
<tr pattern="listItem" view="alternateColor">
<td model="referrer" view="Text">
Referrer</td>
<td>-></td>
<td model="request_resource" view="Text">
Resource</td>
</tr>
<tr pattern="emptyList">
<td colspan="2">There is nothing to display.</td>
</tr>
</table>
</body>
</html>
Alex Levy developed this template, and -- showing better style than I have in my examples -- used CSS2 to control the exact presentation of elements. Obviously, the basic layout of the page is the same with or without the style sheet.
Notice that the <table> element is assigned the View "List," which is a basic Woven View, as is "Text." On the other hand, "alternateColor" is a customized View that we define in code below. Some elements have a pattern attribute that is used by the controlling View to locate matching children. In particular, a List View is composed of an optional listHeader, some listItemchildren (one template tag, but expanded during generation), and an emptyList child in case the Model does not locate any data. These patterns are standard attributes that a List View uses; other Views would utilize other patterns for expansion.
The code for this version of a weblog server creates a custom Twisted server. Rather than update based on requests by clients, we add a repeated callback to the update() function to the server's Reactor; this is substantially the same as withtlogmaker.py in the prior installment. Let's look at the setup code first before we examine the customized Page resource:
Listing 6. WeblogViewer.py custom Twisted server
import webloglib as wll
import os, sys
from urllib import unquote_plus as uqp
from twisted.internet import reactor
from twisted.web import microdom
from twisted.web.woven import page, widgets
logfile = '../access-log'
LOG = open(logfile)
RECS = []
NUM_ROWS = 25
def update():
global RECS
RECS.extend(LOG.readlines())
RECS = RECS[-NUM_ROWS*3:]
reactor.callLater(5, update)
update()
The interesting stuff comes with our customization of the class twisted.web.woven.page.Page. Most of what we do is magic, in the sense that you need to define specially named attributes and methods.
Listing 7. WeblogViewer.py Twisted server (continued)
class WeblogViewer(page.Page):
"""A Page used for viewing Apache access logs."""
templateDirectory = '~/twisted/www'
templateFile = "WeblogViewer.xhtml"
# View factories and updates
def wvupdate_alternateColor(self, request, node, data):
"""Makes our table rows alternate CSS classes"""
# microdom.lmx is very handy; another example is located here:
# http://twistedmatrix.com/documents/howto/picturepile#auto0
tr = microdom.lmx(node)
tr['class'] = ('odd','even')[data['_number']%2]
# Model factories
def wmfactory_filename(self, request):
"""Returns the filename of the log being examined."""
return os.path.split(logfile)[1]
def wmfactory_entries(self, request):
"""Return list of dict objects representing log entries"""
entries = []
for rec in RECS:
hit = [field.strip('"') for field in wll.log_fields(rec)]
if hit[wll.status] == '200' and hit[wll.referrer] != '-':
# We add _number so our alternateColor view will work.
d = {'_number': len(entries),
'ip': hit[wll.ip],
'timestamp': hit[wll.timestamp],
'request': hit[wll.request],
'request_resource': hit[wll.request].split()[1],
'status': hit[wll.status],
'bytes': hit[wll.bytes],
'referrer': uqp(hit[wll.referrer]).\
replace('&&',' &'),
'agent': hit[wll.agent],
}
entries.append(d)
return entries[-NUM_ROWS:]
resource = WeblogViewer()
There are three categories of things that our custom Page does. The first sets up the template to use with this resource.
The second defines a custom View, using the magic method prefix wv (Woven view). All we really do in the custom View is set theclass attribute to one of two values from the CSS stylesheet, to make alternating rows of display have a different color. But you could manipulate the node however you like, using a DOM-like API.
The third category is interesting. We define two Models using methods with wmfactory_ prefixed to the name of the Model itself. Since filename is displayed in a Text View, it is best to return a string. And likewise, entries displayed in a List View should get a list of entries as return value. But what about the Models referrer and request_resource that are used in the XHTML template? No custom methods were defined for these models. However, the listItem pattern that surrounds nodes with these Models has a dictionary made available to it -- the entries dict returned by .wmfactory_entries(). And this dictionary, in turn, contains keys for request_resource and referrer; you do not need a custom method to support a Model, a dictionary with the necessary key works fine. Since the View for the referrer node is Text, said dictionary should contain a string for a value (but if not, Woven makes good efforts to coerce it).
Creating a custom server based on the custom WeblogViewer.py resource works as we have seen before. Create a server, and then launch it:
% mktap web --resource-script=WeblogViewer.py --port 8080
% twistd -f web.tapIn the final installment
This introduction has only scratched the surface of Woven. There are whole labyrinths within that package, but I hope the example I presented gives a bit of the feel for the templating system.
Next time, in the final installment of this series on Twisted, I will pick up some odds and ends, including a brief look at security. We will also take a look at a few of the specialized protocols and servers contained in the Twisted package.
Resources
- Twisted comes with quite a bit of documentation, and many examples. Browse around the Twisted Matrix home page to glean a greater sense of how Twisted works and what has been implemented with it.
- Part 1 of this series covered network programming with the Twisted framework and asynchronous networking (developerWorks, June 2003).
- In Part 2 of this series, we set up a Web server and implemented basic services (developerWorks, July 2003).
- In Part 4 of this series, David looks at specialized protocols and servers contained in the Twisted package, with a focus on secure connections (developerWorks, September 2003).
- A simple version of a weblog server was presented in the developerWorks tip, "Asynchronous SAX" (developerWorks, May 2003).
- You can download the Webloglib module and other things associated with this Twisted series at David's Web site.
- David would like to thank Twisted user Alex Levy, who helped him with the CSS examples in this installment.
- Find more articles for Python developers in the developerWorks Linux zone.
No comments:
Post a Comment