Business documents often require special handling. Enter Electronic Document Interchange, or EDI. EDI is more than simply transferring files using email or http (or ftp), because these are documents like orders and invoices. When you send an invoice, you want to be sure that:
1. It goes to the right destination, and is not intercepted by competitors.
2. Your invoice cannot be forged by a 3rd party.
3. Your customer can’t claim in court that they never got the invoice.
The first two goals can be accomplished by HTTPS or email with S/MIME, and in some situations, a simple HTTPS POST to a web API is sufficient. What EDI adds is the last part.
This article does not cover the messy topic of formats for the files exchanged. Even when using a standardized format like ANSI or EDIFACT, it is ultimately up to the business partners. It is not uncommon for business partners to use an ad-hoc CSV file format. This article shows you how to configure Fedora to send and receive in an EDI setup.
Centralized EDI
The traditional solution is to use a Value Added Network, or VAN. The VAN is a central hub that transfers documents between their customers. Most importantly, it keeps a secure record of the documents exchanged that can be used as evidence in disputes. The VAN can use different transfer protocols for each of its customers
AS Protocols and MDN
The AS protocols are a specification for adding a digital signature with optional encryption to an electronic document. What it adds over HTTPS or S/MIME is the Message Disposition Notification, or MDN. The MDN is a signed and dated response that says, in essence, “We got your invoice.” It uses a secure hash to identify the specific document received. This addresses point #3 without involving a third party.
The AS2 protocol uses HTTP or HTTPS for transport. Other AS protocols target FTP and SMTP. AS2 is used by companies big and small to avoid depending on (and paying) a VAN.
OpenAS2
OpenAS2 is an open source Java implemention of the AS2 protocol. It is available in Fedora since 28, and installed with:
$ sudo dnf install openas2
$ cd /etc/openas2
Configuration is done with a text editor, and the config files are in XML. The first order of business before starting OpenAS2 is to change the factory passwords.
Edit /etc/openas2/config.xml and search for ChangeMe. Change those passwords. The default password on the certificate store is testas2, but that doesn’t matter much as anyone who can read the certificate store can read config.xml and get the password.
What to share with AS2 partners
There are 3 things you will exchange with an AS2 peer.
AS2 ID
Don’t bother looking up the official AS2 standard for legal AS2 IDs. While OpenAS2 implements the standard, your partners will likely be using a proprietary product which doesn’t. While AS2 allows much longer IDs, many implementations break with more than 16 characters. Using otherwise legal AS2 ID chars like ‘:’ that can appear as path separators on a proprietary OS is also a problem. Restrict your AS2 ID to upper and lower case alpha, digits, and ‘_’ with no more than 16 characters.
SSL certificate
For real use, you will want to generate a certificate with SHA256 and RSA. OpenAS2 ships with two factory certs to play with. Don’t use these for anything real, obviously. The certificate file is in PKCS12 format. Java ships with keytool which can maintain your PKCS12 “keystore,” as Java calls it. This article skips using openssl to generate keys and certificates. Simply note that sudo keytool -list -keystore as2_certs.p12 will list the two factory practice certs.
AS2 URL
This is an HTTP URL that will access your OpenAS2 instance. HTTPS is also supported, but is redundant. To use it you have to uncomment the https module configuration in config.xml, and supply a certificate signed by a public CA. This requires another article and is entirely unnecessary here.
By default, OpenAS2 listens on 10080 for HTTP and 10443 for HTTPS. OpenAS2 can talk to itself, so it ships with two partnerships using http://localhost:10080 as the AS2 URL. If you don’t find this a convincing demo, and can install a second instance (on a VM, for instance), you can use private IPs for the AS2 URLs. Or install Cjdns to get IPv6 mesh addresses that can be used anywhere, resulting in AS2 URLs like http://[fcbf:fc54:e597:7354:8250:2b2e:95e6:d6ba]:10080.
Most businesses will also want a list of IPs to add to their firewall. This is actually bad practice. An AS2 server has the same security risk as a web server, meaning you should isolate it in a VM or container. Also, the difficulty of keeping mutual lists of IPs up to date grows with the list of partners. The AS2 server rejects requests not signed by a configured partner.
OpenAS2 Partners
With that in mind, open partnerships.xml in your editor. At the top is a list of “partners.” Each partner has a name (referenced by the partnerships below as “sender” or “receiver”), AS2 ID, certificate, and email. You need a partner definition for yourself and those you exchange documents with. You can define multiple partners for yourself. OpenAS2 ships with two partners, OpenAS2A and OpenAS2B, which you’ll use to send a test document.
OpenAS2 Partnerships
Next is a list of “partnerships,” one for each direction. Each partnership configuration includes the sender, receiver, and the AS2 URL used to send the documents. By default, partnerships use synchronous MDN. The MDN is returned on the same HTTP transaction. You could uncomment the as2_receipt_option for asynchronous MDN, which is sent some time later. Use synchronous MDN whenever possible, as tracking pending MDNs adds complexity to your application.
The other partnership options select encryption, signature hash, and other protocol options. A fully implemented AS2 receiver can handle any combination of options, but AS2 partners may have incomplete implementations or policy requirements. For example, DES3 is a comparatively weak encryption algorithm, and may not be acceptable. It is the default because it is almost universally implemented.
If you went to the trouble to set up a second physical or virtual machine for this test, designate one as OpenAS2A and the other as OpenAS2B. Modify the as2_url on the OpenAS2A-to-OpenAS2B partnership to use the IP (or hostname) of OpenAS2B, and vice versa for the OpenAS2B-to-OpenAS2A partnership. Unless they are using the FedoraWorkstation firewall profile, on both machines you’ll need:
# sudo firewall-cmd --zone=public --add-port=10080/tcp
Now start the openas2 service (on both machines if needed):
# sudo systemctl start openas2
Resetting the MDN password
This initializes the MDN log database with the factory password, not the one you changed it to. This is a packaging bug to be fixed in the next release. To avoid frustration, here’s how to change the h2 database password:
$ sudo systemctl stop openas2
$ cat >h2passwd <<'DONE'
#!/bin/bash
AS2DIR="/var/lib/openas2"
java -cp "$AS2DIR"/lib/h2* org.h2.tools.Shell \
-url jdbc:h2:"$AS2DIR"/db/openas2 \
-user sa -password "$1" <<EOF
alter user sa set password '$2';
exit
EOF
DONE
$ sudo sh h2passwd ChangeMe yournewpasswordsetabove
$ sudo systemctl start openas2
Testing the setup
With that out of the way, let’s send a document. Assuming you are on OpenAS2A machine:
$ cat >testdoc <<'DONE'
This is not a real EDI format, but is nevertheless a document.
DONE
$ sudo chown openas2 testdoc
$ sudo mv testdoc /var/spool/openas2/toOpenAS2B
$ sudo journalctl -f -u openas2
... log output of sending file, Control-C to stop following log
^C
OpenAS2 does not send a document until it is writable by the openas2 user or group. As a consequence, your actual business application will copy, or generate in place, the document. Then it changes the group or permissions to send it on its way, to avoid sending a partial document.
Now, on the OpenAS2B machine, /var/spool/openas2/OpenAS2A_OID-OpenAS2B_OID/inbox shows the message received. That should get you started!
Photo by Beatriz Pérez Moya on Unsplash.
Kefah Issa
Very interesting subject. I really appreciate digging into it.
I am interested in the more generalized application of such concepts. Add to the above the option of having those documents (contracts / invoices …etc) machine readable with proper xml/json schema definitions; where sending an invoice and receiving a payment notification then sending back a receipt could be directly integrated with the accounting systems. This should in the future replace the classical communication tools such as emails.
Stuart D Gathman
All my real life implementations use either ANSI or EDIFACT standards for the machine readaable documents. EDIFACT has the more elegant envelope (the UNA segment is a masterpiece of functional minimalism), ANSI is much more usable as to actual document definitions. EDIFACT feels like the committees failed to reach consensus and compromised with alternatives (you can do it this way, or that way, or this other way…)
There is no “in the future” – machine readable EDI has been in wide spread use since the 80s. My oldest official documentation is from 1992. If you simply can’t wrap your head around the simple flat record syntax, there are standard bidirectional mappings to/from ANSI/EDIFACT to/from XML, JSON, or YAML. The newer formats are functionally equivalent. You certainly don’t want to redo the document definitions.
So your business partners are already going to be using ANSI or EDIFACT. If you can’t work with the original record syntax directly, map to JSON or YAML (Please no XML! 🙂 ) You can use your own mapping if you want – as long as you have test cases to make sure it is bidirectional. The mapping needs to know which segment (record) types are subsidiary (nested under another type), but no other specific knowledge. You should have machine readable definitions of the data elements and segments for ANSI/EDIFACT – as this allows checking valid ranges, identifier sets, number vs alpha, etc for outgoing documents so as not to embarrass yourself. 🙂
Stuart D Gathman
If you are only going to import your XML/JSON to a DOM (document object model), then there is absolutely no point in translation. Just import the ANSI/EDIFACT document directly to your DOM, and output directly from a DOM.
Kefah Issa
Many thanks for the elaborated response. very insightful.
I have previous exposure to EDI /ANSI X12 as I worked with Transaction sets in healthcare (835/837 et al) / HIPAA.
After reading your comments I digged more into EDIFACT and can indeed see the various aspects of commercial transactions being covered.
Thank you again for sharing.
Stuart D Gathman
The US CBP still uses an EDI format called CATAIR that was designed for 80 column punched cards. It is used to report the contents of shipments going in and out of the country for statistical reporting and paying of taxes and duties. (Like US income tax, the self reports are enforced via random deep inspections of the actual cargo.) There are no separators in CATAIR – fields are identified by card position, and are coded in EBCDIC. It is still functionally equivalent to later formats. I generally translate to ACSII in an ANSI like syntax with separators – to avoid storing all those blank spaces.