As with all dynamic content, the idea behind servlets is that HTML content should be generated at runtime by a process running on the server, rather than simply being copied from a static file. This enables the content to change depending on the context of the request, such as the state of a database, the time of day, http request parameters, etc.

Servlets in particular are java objects that extend javax.servlet.http.HttpServlet, and implement methods which return HTML code in response to HTTP requests. These objects are run as threads inside a servlet container (or engine) - which may either be a sub-process of the web server, or a completely independent process. Servlets are similar in concept to CGI processing, but have two key advantages. The first is that they are instantiated as threads, rather than requiring the spawning of a completely new process, which means a big saving in server resources. The second is that data persistence is handled cleanly:

When a .jsp page is requested it is compiled into a servlet by a process running within the web server servlet container. The compiler parses the template, translating its contents into the equivalent Java source code which, when executed, will generate the output indicated by the original file. Static HTML is translated into Java strings, to be written unmodified into the output stream, and JSP tags are translated into Java code for generating dynamic content. (Nb. the page is only compiled once; subsequent requests invoke the ready-compiled servlet.)

It is often claimed that an advantage of JSP is that it cleanly separates presentation (which is handled by the html template) from data (which is generated from the java fragments). This is not true in practice, since presentation logic tends to be intimately mixed with data. The result is that JSP pages tend to be messy, with no clear division between Java and HTML.

<jsp:usebean id="myCustomer" class="CustomerBean" scope="session">

If no customerBean yet exists in that session then the container will create one - and it will then persist beyond the life of a single request. The same CustomerBean can then be referenced by all subsequent requests within the session.

Velocity is a templating/scripting language designed to be a cleaner and simpler alternative to JSP -- basically a more Java'ish version of Webmacro. It's now reached v1.0 and the documentation looks quite thorough. But does the world really need another scripting language?

Struts and Turbine are servlet-based frameworks for building componentized web applications. (A framework, in this context, means a set of cooperating classes that make up a re-usable design for a specific class of software. A framework rigidly dictates the architecture of an application; therefore they can save a lot of effort if they work well, but can be a nightmare if not well-designed.) Turbine and Struts are aiming to build a library of reusable components, but it is not yet clear if either will reach critical mass.

<ul> #foreach $customer in $list { <li>$customer.name lives at $customer.address } </ul>

Sun claim that "Java + XML = portable code + portable data"; in other words, treat XML as a platform- and application-independent way of storing and communicating data. It is certainly possible, for example, to transform XML content into an HTML document, or even browse it directly using an XML-enabled browser, such as IE5; but XML is, first and foremost, a way of defining data stores such that they can be flexibly manipulated. In this respect the nearest historical equivalent to XML is not HTML but CSV (Comma Separated Values) - the convention of storing information in a flat file, one record per line. The difference between the two is that XML can define far richer structures than a simple list of vectors (namely, recursively-defined trees with weakly-typed node elements).

Consider this motivating example. Suppose we have a large collection of complex structured documents - with titles, introductions, summaries, modification dates, authors, paragraphs, and so on - on which we need to perform three kinds of operations. First, we want to execute complex searches (all documents modified more than once, all documents that refer to ones written by me, etc). Second, we want to present the results in a variety of html formats (full document, summary, outline, etc). Third, we want some content management functions (transferring to production, access by workflows). XML would be the canonical way to store the documents such that they could be accessed in all these ways.

Of course each of these accessor functions could be optimized in various ways - searches could be optimized by using indexed DB tables, presentation could be achieved by storing pre-compiled static html files, etc - but XML would still be the most convenient master data store. Alternatively, XML could be used as the standard for communicating data, rather than storing it. Oracle, for example, provides very good support for getting XML in and out of an RDBMS, so we could use this to store the raw master data, but it may still make sense to pass documents to (and from) the accessor functions in a common XML format.

The W3C-standard mechanism for transforming XML is XSL (eXtensible Stylesheet Language), which has two major components. The first is the Formatting Objects (XSL:FO) standard, which is a hardcore formatting standard for converting XML data into binary formats such as PDF, MSWord, etc. If you need to worry about XSL:FO then you are in the wrong job.

The second component of XSL is XSL Transformations (XSLT), which defines a syntax for converting XML documents into other plain-XML documents. XSLT is a recursive declarative language (like Prolog), that itself is written in XML. So, in the case of our previous example, suppose we wanted to take a raw XML document and present an outline of it in pretty HTML. The XSLT document would say something like:

1. If you find a chapter element, then print out its title wrapped in <H1> tags and an <href>, and process the rest of its contents.
2. If you find a sub-section element, then print out its title wrapped in <H3> tags, and skip the contents, etc.
   

SAX uses an event-based parser. You define a reader object that includes a number of call-back methods corresponding to the bits of content you are interested in; the parser then crawls over the document, and calls those methods whenever it hits a bit of corresponding content. A typical application would be a servlet that uses repeating elements in an XML document as a simple database - the SAX parser would crawl over the document, generating content for the servlet to include in an HTML template.

DOM, on the other hand, uses a tree-based parser. As its name suggests, a DOM parser takes an XML document and constructs a tree of Java (or C++, or perl, etc) objects whose structure and content reflects that of the original document - one node object per XML element. One use of DOM would be a servlet that takes an HTML template (i.e. an XML template that conforms to the HTML DTD), inserts bits of dynamic content, and outputs the result as HTML.

Note the difference between SAX and DOM in these two applications. A SAX parser crawls over an XML document invoking call-backs as it goes, then dies. It is therefore suited to parsing XML documents that are liable to change, such as those that act as data stores or the results of DB queries. A DOM parser, on the other hand, generates an object structure that can persist as long as you want; and so is more suited to parsing XML documents that are relatively fixed, such as templates.

Cocoon seems a simple, robust, and popular framework for publishing XML using XSLT; and a number of live commercial sites are using it in this way already. However there are still questions about using it as a full-blown publishing framework. The first is that it is very new, with very little documentation. Version 2 is about to be released, but the early announcements for this include the worrying admission that version 1 was just a "proof of concept". The second question mark is around XSP - which is supposed to be the centrepiece of Cocoon 2.0. Not only is XSP complicated and not particularly well-documented, but also seems to have a deeper conceptual problem:

XSP, as the name suggests, is the XML version of JSP; in other words it mixes static XML data with Java fragments. Now this combination of static stuff and dynamically generated stuff makes sense if the resulting page is HTML, since HTML, by its nature, combines static presentation with dynamic content data. But XSP generates XML pages, and XML documents are only supposed to contain data, with presentation being handled by XSL or a template. The question is, why mix static and dynamic data to generate a single page of XML data? If it is fast-changing data then stick it in a database (or persistent servlet objects) and use servlets to grab it when requested. If it is slow-changing data (such as text for articles) then store it in a XML document and generate cached HTML pages using XSL. But combining these datastreams only to store the result in an XML document just seems to threaten data synchronization problems. (Maybe I'm missing something, but XSP seems to be a technology in search of an application.)

For a simple application, it is just about possible to fit both the View and Controller functions in a single servlet or tcl script, but it is a tight squeeze and three pressures make it worse. The first is that web sites have to do more and more processing. For example, instead of simply serving files and storing forms they may now have to personalise those files and send those forms through credit-card validation. Second, instead of having a dedicated database, the web site may be accessing a legacy database that is also serving the rest of the business - which means that getting appropriate data safely may be more difficult than a straight SQL query. The fulfillment DB for a web retail business, for example, may be sat in a warehouse in Leicester, and can only communicate with the webserver host via ftp. Third, there may be more clients than just an html browser - such as WAP or an XML client - each of which server similar business functions but in very different ways.

XMLC is the reason why Enhydra is so popular. It takes a flat HTML (or XML) file and generates a Java class that, when called by the multiserver, generates the HTML corresponding to the original flat file. These presentation objects implement the DOM API, and include methods for defining content dynamically via business objects with implicit objects available for storing persistent session data. Most popular RDBMS are supported via JDBC. The nice thing about XMLC is the way that it generates Java objects from (almost) raw HTML. But the down-side is that you have to add code to these objects which will be lost every time you re-compile a change in design. So XMLC would seem to work best when you either have very stable designs or programmers who don't mind working around designers. XMLC is now available for stand-alone use.

Some things, such as repeated elements (tables) and forms, are clunky to implement using XMLC; nonethess it is being used on a number of live sites, and the commercial backing means that it is relatively well documented - complete with endearing animal mascot (in this case Ollie the Otter). In addition Lutris seem genuinely commited to the open source framework, and to developing the system to include new technologies in the future; and the response to email queries has been impressive (less than half an hour in one case.)

Struts

An MVC framework.

Avalon

Framework for designing components and services, adhering to a set of design patterns. Aimed at server-side development. Used by Cocoon 1 and 2, James, and others.

Cocoon 1 and Cocoon 2

A servlet-based publishing platform built with Avalon. Developer and user mailing lists. Includes XSP and ESQL logic tags (and more), XML/XSL support via Xalan/Xerces. The OPP demo was programmed in 4 days by a novice, in an ACS way: creation of a set of independent pages calling various APIs, without any thick Avalon service/module. This approach seems suitable and very efficent, for small projects.

Turbine

A servlet-based framework.

JBoss

An EJB container and framework.

Open Symphony

An EJB-based framework.

ACS Java

Currently in 4.6, which is mainly about lower-layers of the architecture.

Documentation: doc, business doc, and UML.