CHAOS and IRIDIUM (by alaric)
I came accross a copy of AI Memo 628, which describes the MIT CHAOS network. I love reading about old computer technology, since they come from a time where creativity was unbridled; nowadays, suggesting a new technology that isn't intimately compatible with The Legacy Systems is unwise, so we carry their mistakes onwards for ever more.
As it happens, I was pleased to find that CHAOS had a particular feature I'd independently come up with for my own IRIDIUM protocol design: it supports both connection-oriented and connectionless communication, but unlike IP's distinction between TCP and UDP, the two are integrated.
In the IP world, if you choose to use UDP, you have a non-guaranteed potentially-unordered non-flow-controlled message-based protocol. If you choose TCP, then you have to perform a three-way handshake to establish connection state between client and server before you can even state your request to the server (so the three-way handshake may be in vain, if your request is erroneous). Having set up that connection, you now have a flow-controlled ordered-stream connection with guaranteed delivery; if you want packet boundaries in that stream, you need to put your own markers in.
Sometimes you have an application that demands flow-controlled guaranteed transmission of messages without care for ordering; in which case you can go the easy route and use TCP, and lose performance due to TCP's efforts to provide ordering and deal with the complexity of your own message-boundary system in your protocol, and the three-way handshakes; or you can go the hard route and implement your own flow control, retransmission, and fragmentation on top of UDP. Either way, you're having to make tradeoffs.
But on the CHAOS network - or, hopefully, one day with IRIDIUM-over-UDP, over the Internet too - the same protocol simply offers different types of service. The principle is simple: you send your message packet, containing the details of your request, to the destination host. And it then responds with an answer packet. Which is just like UDP, except that the network stack handles retransmission and flow control for you - but if this initial request is actually the beginning of a connection, then the equivalent of a TCP handshake to set up connection state is also carried along with the initial request/response.
The neat thing is that the server process gets to decide whether to accept or reject the connection request - so if the initial request packet contains things like authentication details and the name of the resource being requested, then these can be checked and the entire initial connection rejected outright without needing to establish session state or back-and-forth SYN packets. So the client gets a quicker rejection, and less server and network resources are consumed, reducing the effectiveness of flooding attacks.
It also enables protocols to automatically scale depending on message sizes. If the request and response are small enough to fit in single packets, they can; but if the client has a large request, it can send the most important fields of the request message in the request packet, along with a flag noting that it has more to send - and the server can then, if it likes the request header, consent to open a connection to receive the rest. Likewise, the server can decide whether to include its response message entirely in a response packet, or instead just include the start of the response along with connection state for receiving the rest.
If HTTP were implemented on top of something like that, we'd have a snappier Internet; HTTP requests that result in small pages and images (or redirects or errors) could be handled without a three-way-handshake, and even for larger requests (such as POSTs) and large responses, actual data transmission would already be in progress as the connection is being set up, so it would get started more quickly. There would be less latency.
Also, CHAOS preserved message boundaries, and made them visible to the application; if the application was doing something like SSH or streaming audio then it's free to ignore the boundaries and just consider it to be a stream of bytes, but the application can use the message boundaries if it wishes. This does have one downside, though - applications need to be aware of a maximum size of packet the network can take (the MTU), and make sure their messages are split up if they're too big. This was bearable for CHAOS since the network could enforce an MTU from end to end, but over the Internet, the MTU can be anything from 68 bytes to 64KiB, and can only be found out dynamically (and can change at random). So for IRIDIUM, I've isolated the application from needing to know about that by not forcing each application message to fit into a single network packet; quite apart from the application-visible connection state to establish a stateful dialogue between client and server, IRIDIUM may establish lightweight connection state to transmit large messages over multiple packets (and I want to use a protocol more like NetBLT than TCP for this, as it should be more efficient).
However, it would be drifting beyond my original mandate of comparing with CHAOS to start talking about how IRIDIUM distinguishes requests and responses in either direction along a connection so that responses to requests from client to server are distinguished from asynchronous 'callbacks' from server to client (as opposed to TCP's "stream of bytes in each direction, roll your own protocol"), or the fact that within a connection the same logic of messages being able to carry connection-opening state still follows so that you can have sub-connections within connections, with all the sub-connections get closed when the connection closes (think about opening an ssh connection to a remote host, then initiating multiple SFTP file transfers and forwarded X11 connections and the like), or the support for pluggable connection encoding state to handle encrypted and/or compressed connections... so perhaps I'll save that for another blog post.
Fri 10th Jul 2009 10:35 am 
Subscribe