When parties on the Internet need to exchange documents, they often end up sending it as an email attachment. Although this works for documents up to a certain size, it has a few serious drawbacks:
- There is a limit to the size of documents that can be sent;
- Acknowledgements are hop-by-hop rather than end-to-end;
- Documents must be static, and may not be sessions or similar interactive things.
The AMQP mechanism is designed as a method for sending documents with a format description from a sender to a recipient. The format description and sender/recipient infomrmation is helpful in locating the right Input Queue, or other form of handler within the Reservoir.
The format description can be a MIME-type or, alternatively, a URI so that we may even make informal extensions like http://types.arpa2.org/whatever or even other-technology URIs like urn:nfc:ext:arpa2.org:whatever -- meaning, we can extend the typing discipline without grabbing something that isn't ours.
Check out a few example use cases.
How to send over AMQP
AMQP is agnostic about the format of sender and recipient addresses, but it does support them. A hosting solution can listen to standardised TCP ports 5762/5761, comparable to ports 80/443 for HTTP service. The protocol can be initiated with a handshaking step that switches from plaintext to TLS and/or GSSAPI. (In addition, it appears to be possible to use DNS SRV records to point from under a domain to a service for amqp or, if the STARTTLS procedure is considered less desirable, for amqps.)
The AMQP scheme for URIs has not been standardised yet but has a proper definition and a more detailed one in line with the general standard. Indeed, it does support addressing firstname.lastname@example.org style targets.
AMQP itself is a wire-level protocol, and does not seem to object senders and recipient addresses formatted in email@example.com form, like the DoNAI form that we use throughout the ARPA2 infrastructure.
Connections can be initiated as plaintext (the so-called STARTTLS approach) and then upgraded to TLS or GSSAPI; this means that TLS-KDH can be used, as well as direct Kerberos-based access. Both are very, very powerful mechanisms for fast cross-realm connectivity.
You would submit your document to an address such as firstname.lastname@example.org and the user john of the domain name example.com would be able to pick it up from a queue that may be looking for your input, or perhaps any input at all, or just input of a certain type.
The precise mechanism that we are considering upon arrival of an AMQP message is to retrieve a recipient's Input Queues, select those that match with the type of the message body (allowing wildcards such as image/* as well as Input Queues with multiple acceptable formats) and deliver the message there. When no suitable Input Queue is found, the message will be rejected.
During delivery to an Input Queue, an object may be placed in a spool or in the Object Store, the storage URL is then used to setup an entry in the Input Queue. Usually, the Input Queue will not be stored in LDAP yet, so as to avoid overzealous writing into this infrastructure. TODO: Different Backend?
Processing the new entry may now commence through a variety of mechanisms, both manual and automatic; among the options is placing the object in a Resource Collection within the Reservoir.
When connecting to a remote AMQP user email@example.com, to send a message from firstname.lastname@example.org, we perform the following procedure:
- If a connection already exists between example.com and example.net, reuse that and skip the next 3 steps
- TODO: Lookup the DNS SRV record for the target realm example.net by looking for the labels _amqp.sctp.example.net and/or _amqp._tcp.example.net; note that SCTP can be helpful in multiplexing traffic; default to host example.net:5762 when no SRV records are found
- Connect to the host:port combination found, and set up for GSSAPI; failing that, we may care to try again using TLS-KDH
- Use the local Kerberos identity amqp/example.com@EXAMPLE.COM and expect as a remote identity amqp/example.net@EXAMPLE.NET -- where the realms may in fact be different as per the _kerberos TXT records in DNS.
- Offer the Resource in the Output Queue; provide its MIME-Type in a Content-Type header, fill out the Subject if possible, and set Message-ID appropriately, set User-ID to email@example.com and set To to firstname.lastname@example.org; possibly supply more headers
- A local node will always identity as a service/host identity, where the service is always amqp and the host represents the domain name in the local-end identity; a remote node is verified to live by the same rules.
- In terms of AMQP, the To header defines the target node, and behind that is a set of named Links that we refer to as Input Queues in terms of the Reservoir. In terms of message switching facilities, we match headers and will usually be specifically interested in the To and Content-Type headers, as well as the Link Name for the given To recipient.
- Given that a receiving end is identified, the message itself should be exchanged and acknowledged. It is up to the Input Queue policy when an acknowledgement will be generated; this may be upon correct storage in the Input Queue, or it may be when the user actively processes it somehow, for instance by moving it to a Resource Collection in the Reservoir.
This section gives a few alternative interfaces for the exchange of messages or documents with the Reservoir; they are generally considered part of added services through the ServiceHub phase of the InternetWide Architecture; the AMQP service is a more general, service-agnostic facility that can be used in various ways in such services. This includes the facilitation of Input Queues and Output Queues.
There are more push-style algorithms that may work alongside AMQP to achieve the same goals:
- Document uploading web services such as FileSender are eminently suitable to help web browser users to an upload facility;
- Various file sharing mechanisms may be used for uploads;
- AFS and CIFS might be provided for Kerberos5-authenticated uploads.
There are also alternate ways of downloading documents from authorised submitters:
- Document downloading through a cloud system such as OwnCloud (though it has a long way to go before it recognises the welcomed Identity of an ARPA2 user;
- ATOM or RSS can provide REST document downloads directly from the Object Store;
- AFS and CIFS are filesystems that can be locally mounted, and it can hold dropboxes that your friends can send to;