Protocol agnostic middleware (by alaric)
It strikes me that perhaps one of the reasons that middleware hasn't taken the world by storm is that middleware tends to require some inconvenient protocol to access it.
Even the Web Services folks with SOAP and friends don't seem to have changed the world much, for all the effort of choosing a protocol that clients can be knocked up for easily from scratch in most environments.
However, in my work for Frontwire, I've ended up creating a server that does things in response to:
- Java RMI calls
- Work for it to do being directly inserted into an SQL database
- Scheduled events
- HTTP requests
- Incoming emails
- FTP uploads
What does that tell us about currently available middleware "solutions"? Originally, we used an EJB container to handle the incoming RMI, and hacked everything else on; now, in the interests of making it lighter weight (and thus faster and less buggy) it runs as a normal RMI server, that happens to spawn off threads and have supporting applications to handle the other protocols.
All of these different interfaces into our code were needed for one reason or another - mainly due to the constraints of the client code connecting to it. The SQL work queue came into being to allow PHP scripts on an adjacent server to invoke actions - they didn't happen to need any results passed back, so a queue was chosen, allowing the PHP to get on with other things while the server gets round to servicing the request when it's ready. If the server is down or busy, the request waits silently in the database.
Third party systems that interoperate with us either send emails, or upload files via FTP, or fetch information via HTTP. We have yet to have anyone ask for a SOAP, XML-RPC, or even REST Web Service interface to our software, but it may happen.
It has struck me that perhaps this is what middleware needs to be truly useful. It's easier for me to write my PHP client to poke work into a queue table in an SQL database than to make it perform a SOAP invocation - SQL client libraries were already installed and a SOAP client wasn't, for a start. A few other parts of the application do use HTTP in an RPC-like manner, but without any of SOAP or XML-RPC; I just POST to a URL with the relevant parameters, and then read the response.
Modern middleware systems provide at most a few invocation protocols - many now support SOAP, as well as another such as CORBA or DCOM. They market themselves on scaleability and features like transactions, clustering, manageability and so on.
How about a very lightweight middleware system that instead prides itself on being accessible from everywhere? It could support raw HTTP, XML-RPC, SOAP, Java RMI (even if it's not a Java application itself, potentially - RMI is just a protocol; it's possible to be an RMI client or server without being written in Java), email, raw TCP and UDP sockets, IIOP (the protocol CORBA uses over TCP/IP), DCOM, ONC RPC (the protocol on which NFS and NIS are based; client libraries come with pretty much any installation of Unix, so although you've never heard of it, never forget that it was far, far, wider reach than most other RPC protocols around), FTP, and even SQL via queue tables.
Not all of the protocols can be transparently mapped to a single "remote procedure call" abstraction, of course. The actual business objects will need to provide extra information to, for example, be accessible via raw UDP - perhaps the middleware system can react to the presence of an exported procedure called udp_PORT that accepts a byte array as its sole argument and returns a byte array by binding to the specified UDP port, passing incoming packets to the procedure, and if it returns non-null, sending a reply packet.
However, protocols designed for remote procedure activation (RMI, DCOM, IIOP, XML-RPC, SOAP, ONC RPC) can all be automatically provided from a list of exported procedures and their argument types, with a little jiggling to deal with the fact that ONC RPC doesn't have an inbuilt representation for linked lists while RMI does (java.util.List), and so on.
Such a middleware platform, that struggles to make the business logic within it as visible as possible, would free enterprise developers to use the most appropriate language for each client application, rather than constraining them. It would make it worth incorporating more business logic into the middleware layer, where it can be shared, rather than creating many isolated ad-hoc systems on top of the middleware, with duplicated functionality.
Tue 3rd Aug 2004 7:47 pm 4 Comments
Other Links to this Post
-
Snell-Pym » Middleware — Mon 4th Jun 2007 @ 10:12 am
-
Snell-Pym » Too many projects — Sun 27th Sep 2009 @ 9:42 pm
RSS feed for comments on this post. TrackBack URI
Subscribe
By Ben, Wed 4th Aug 2004 @ 12:14 pm
All very well and good, but won't each protocol end up supporting only the lowest common denominator of all the other protocols? Otherwise you'll end up writing bindings for all services supported, instead of just one service definition. It's the "little jiggling" that you hint at which is going to be the major problem.
If there isn't a neat way of writing one easy definition for every protocol which supports everything you might want to use, all in one go, then you might as well just implement a private interface which runs on localhost only, and write all the bindings yourself using the pre-existing tools for each protocol. As is done now.
Nice idea though. It would be rather handy.
By alaric, Wed 4th Aug 2004 @ 12:47 pm
Well, for a lot of simple exported procedures that fall within a range of "obvious" types - integers, strings, floats, lists, sets, dictionaries - mappings to all of the main RPC protocols can be auto generated.
For things not intended to be an RPC protocol, it gets more interesting. You can take two approaches (and I would be inclined to offer both). For example, with SMTP, there could be two interfaces: One where the client obeys the middleware server's conventions, and one where the opposite applies.
The former interface would involve sending email in a format specified by the middleware service. Eg, a message to
[name of procedure]@[server]with the body being plain text and containing zero or moreparam=valuestrings, with a defined syntax for the textual representation of each of the types supported by the middleware server (perhaps the XMLy thing SOAP uses). This can invoke any exported procedure. The return value of the procedure is serialised into text and sent as a standard email reply.For the latter interface, if the exported procedure is specially marked as being an SMTP command handler - through metadata, through a naming convention, or just through having a single parameter of type "SMTPEmailMessage", then the message itself is not parsed in any way; it is handed direct to the SMTP command handler to process as it sees fit. This is for use when you do not control the format the client sends the messages in.
In some cases, you will have an RPC protocol that supports some type or concept that is not really supportable by others - in RMI, one could have an instance of a Swing class as a parameter, and supporting that in a non-Java environment would be untenable. Support for such things can only be provided by having special protocol-dependent mappings; if the middleware were written in Java, then exported procedures with Java-dependent signatures would only be available via RMI and not via SMTP.
So there's no need to disallow features that can't be made available over all protocols; and only when a procedure is needed that cannot be accessed by any protocol available to the client under development due to such a feature restriction would anybody need to worry about manually writing protocol mappings on the server.