Rob Huffstedtler rob@bagpipe.com recently wrote a proposal calling for an Open Source Messaging System. Rob compared the idea to Yoga http://samba.anu.edu.au/yoga/, but missed comparing it with Casbah http://www.ntlug.org/casbah/, a project with similar goals but very different ideas about how to get there, and one with a 6 or 7 month design head start. In his proposal, Rob identifies several components of a messaging system; in this article I will attempt to show how Casbah could be used to support or implement these components.
Casbah is a portable, cross-platform, Open Source project. A user's perspective of Casbah is available in Kendall Clark's A Gentle Introduction to the Casbah Project.
The core idea of Casbah is a hierarchical, virtual namespace. The virtual namespace is patterned after the virtual file systems used in modern operating systems, but the Casbah namespace extends into files in a file system or to objects in object databases. Just as web servers are making all sorts of files, objects, and services available through URLs, so does Casbah make objects and services available to scripts and clients. Casbah makes these namespaces available through ``namespace modules'', modules that are loaded explicitly to access file or network services or automatically to extend into known file formats, like XML or mail and news messages. One of Casbah's primary namespace modules is the Coptic persistent object database. Coptic's object database is used to keep application data, server information, scripts, user preferences, and other information users or developers want to keep across script or session boundaries.
A Casbah Server surrounds the storage modules and provides an environment for serving clients, executing scripts and programs in multiple languages, and handling session authentication. The Casbah Server is a long-running, multi-user server that is available to run scheduled requests, respond to email, HTTP, or remote object requests, and run procedures and batch jobs initiated by users, such as updating a web site. The Casbah Server implements a highly dynamic object model to interface between the storage, language, and client communication modules.
The Casbah Architecture was designed with these types of applications in mind:
The following paragraphs show how Casbah might be used to implement the ideas Rob outlined in his paper. Rob's original proposal ideas are quoted.
The system should use iCal, vCard, and other open standards for transmitting message objects such as scheduling information and contacts. This will allow us to interoperate more easily with existing systems and save us the trouble of defining our own format.
Casbah's virtual namespace supports ``reaching into'' container objects to get access to the objects contained within. Using just vCard as an example, Casbah could support locating vCards in the MIME content of mail messages in your system mailbox, news messages from an NNTP server, an archive of mail or news messages, or a tarball of a phone directory. You could also maintain vCard objects as an address list in the Coptic object database and pass vCard objects as local variables or to remote servers.
Public Folders The key thing that differentiates a messenging environment from simple email is the ability to share resources such as address books,schedules, emails, and custom forms. If we could do this with just emails and attachments, it would be a big improvment over the current situation.
Casbah supports the idea of public folders on two levels. Casbah's built-in database supports Access Control Lists (ACLs) on all of the objects contained within it, so you can set up areas within the database for storing objects, emails, attachments, etc., as well as controlling who is allowed to add new objects and who is allowed to view or modify the objects stored there. In addition, using the virtual namespace you can create public folders using already familiar services, such as local or network file systems, HTTP and FTP servers, or other databases.
The system should use an existing SMTP MTA, if possible. Both sendmail and qmail have the advantage of being widely used. They will be more secure and more stable than an alpha-quality MTA built only to handle our needs. They should only be replaced if they prove incapable of supporting our needs.
System mailboxes, local mailboxes and archives, IMAP, POP, NNTP, can all be supported with Casbah storage modules. Outgoing messages would use the most appropriate channel. I compare this capability with the Emacs Gnus news/mailreader. Additional MTAs can be supported to provide special functions if necessary.
Standard mail filtering utilities, like procmail and sendmail aliases, can be used to direct messages to scripts that reside in Casbah servers.
The system should have gateways for proprietary messenging formats such as Exchange, Notes, and Groupwise. This will be necessary for people who want to run a mixed-environment shop.
This author notes that gateways are always difficult beasts to tame. That said, gateways could be implemented as storage servers, where the Exchange, Notes, and Groupwise servers appear in the Casbah namespace and can be manipulated directly, or as remote services where objects are exported from and imported to Casbah namespaces.
Support for LDAP This will bethe best way to take advantage of authentication in the future.
LDAP is starting to be used to hold many types of an organization's network and services information in a hierarchically searched, distributed manner. In addition to the possibility of accessing LDAP as a storage server, the Casbah developers are also investigating versatile authentication services, such as Pluggable Authentication Modules (PAM).
Server Replication We'll need some facility to mirror a server, or portions of a server, so that public resources can be shared in a multi-site network, without everyone using bandwidth hitting a single server.
For simple mirroring, such as replicating centrally maintained resources like core documents, address lists, etc., the easiest way is to use the file system storage module and standard HTTP, FTP, or file system mirroring, or use packaging, like RPM. This solution is extremely robust because it uses utilities that are familiar to most people and is verifiable without special tools.
A new facility would still be needed for more complex situations, such as multiple sources and workflow documents.
Encryption Using ssh and tunneling through it would seem the most obvious solution. Ssh should be embedded in the clients so that the entire system remains easy enough for a secretary to use.
Link encryption has always been a requirement in the Casbah Project. We are working on layered streams which could support SSH or SSL, using any common ciphers.
Forms All clients need to be built with an embedded scripting language which can hook into an API on the server to trigger events such as schedule entries, emails, etc. This is going to be the hardest part, because it requires the client to have an embedded GUI-builder. Right now, it doesn't seem like there are any good ones. Also, we need to define which API to use. Should we build one from scratch, do a black box port of MAPI, or what?
The front-end GUI has been a topic we've been discussing for quite a while. The most promising cross-platform solution we see others developing are the XML and browser front-end renderers, like Mozilla's New Layout -- the ability to layout a complete page to specification, and be able to modify it using even just JavaScript would be a great accomplishment.
Other solutions include Java-based namespace browsers (similar to a File Manager), tailored object viewers (like JavaBeans), and generic form or property viewers using an XML-based GUI language.
[...] Which language to use? Scheme/Guile - popular with the GTK folks, don't know if any Mac or Windows people would wnat to learn it to build apps on the client; Perl - Not really a great language for doing object-oriented GUI stuff; Python- I don't know anything about it -- anyone?; TCL/Tk - Please don't make me go there; Visual Basic - It would make it easy to bring Windows folks into the fold, but it doesn't seem to be a very well thought out language.
After deciding that we wanted Casbah to be language agnostic, we settled on three methods for supporting multiple languages, allowing different schedules for implementation and different levels of integration with the Casbah namespace.
The simplest method is standard client/server communication. Casbah supports a communication protocol that allows any client language to get and put values into the Casbah namespace and to invoke procedures in the server. We've prototyped a plain-text HTTP implementation and a layered streams binary implementation. We also expect a CORBA implementation as well.
The other two methods are closely integrated with the Casbah server. When a client asks the server to invoke a procedure, the server uses a language interface to have a language runtime perform the procedure. The second method for supporting multiple languages uses existing Python, Perl, and Tcl runtimes linked with the server either directly or through a shared memory interface. The third method we are developing is a Scheme-based language runtime with compilers for multiple languages, starting with Perl, Python, and Tcl. By implementing our own compiler we can more closely integrate Casbah namespaces and XML into language expressions.
We also need to tackle the issues of getting people to write the Mac, Windows, Solaris, BeOS, Amiga, PalmPilot, etc. clients, and what the client features should be.
The general consensus seems to be to write the ``big'' client in Java for portability. The Casbah server is written in C using Mozilla's NSPR library and should be portable to all Unices, MacOS, and Win32. Active projects are underway to port NSPR to BeOS and OS/2 as well.
Casbah is designed to be a flexible framework for developing scripted applications, including distributed applications. At this time we do not have anyone working specifically on messaging modules like the ones described above and we would welcome anyone who would like to head up the team identifying the modules to write and organizing the implementation. We are just now making our third development release of CaServer available and are looking for individuals to help with C coding of CaServer components as well as fleshing out the design of modules to help validate the overall design.
Casbah is being actively discussed on the Casbah mailing list and in IRC, we've recently set up a live document web page (Casbah Swiki), and we are planning on having drafts of our protocols, file formats, and APIs available in RFC-like format shortly. These are all available off the Casbah home page.
- The Casbah Project
casbah@ntlug.org