Sunday, April 13, 2014

Practicalities of EDXL - the Sahana Case

Sahana EDXL Experience

"Was EDXL-DE ever used in any of the Sahana Disaster Management Software Products?” "Yes in  EDXL-RM” (Resource Message); see the code snippet that includes EDXL-DE. EDXL-RM is specifically designed as payloads of the EDXL-DE. The development was for a client wanting to interoperate with WebEOC. ”The WebEOC implementation was being handcrafted in raw JavaScript and Eden’s native S3XML was seen as a simpler solution by the Client's own Software Developers, who were handling that side of the interface”. Further work on the EDXL-RM with the EDXL-DE wrapper was stopped until any new use-cases emerged.

“At the National Library of Medicine (NLM), mainly at my behest, they do use EDXL-DE 1.0 as a wrapper.” It encases the triage data (text and photos) sent from the TriagePic applicationto the web site via web services.  This was somewhat as a foundation for and in anticipation of use cases emerging for data interchange with state and local agencies, as well as FEMA.  These use cases have been slow to emerge in practice.  For app communication to the mothership, the overhead of the wrapper, while small compared to photo payloads, is still hard to justify if there’s no pay off. The NLM Webmaster is suggesting that alternative lightweight (but non-standard) wrappers and payloads, using JSON/REST more than XM, is the way forward

“Sahana Eden haven’t updated the existing DE-1.0 wrapper to DE-2.0 and I’m not sure they will invest much more into EDXL because every time they offer EDXL, clients find it way too complex and too narrowing to build their applications upon, and use either our native S3XML format.”

S3XML, like OData, does not set any limits to the contents at all. Furthermore, there doesn’t seem to be a coherent resource “CAP message“. If there was, even the standard REST controller could export the full message. Nevertheless, it would be possible to define such a resource with some minor tweaks to the existing data model. As is, many Eden modules have the tendency to define incoherent data models, yet models can be changed to provide better interoperability.

Particular Problem

The idea with the one particular project was to use Eden as database for managing resource requests, and to have an already functioning WebEOC solution sending those requests to Eden. The desire was to use EDXL-RM for data exchange. The first software build provided an EDXL-RM interface for it.

“A noteworthy issue was that the base data, e.g. requester information (delivery sites, contact information etc) was stored on the Eden side. EDXL did not provide any elements whatsoever to look them up, let alone to maintain them. Nevertheless, that is not an uncommon situation at all: why should the references in EDXL-RM be decentralized? Isn’t it more common that the request management database holds both – request and requester information?”

Realizing the Scope of EDXL-RM

One should not confuse with messaging from information management, where latter is what the project intention were. One should realize the scope of EDXL-RM. As stated in Section 1.3 of the specifications document, it is merely defines 16 separate and specific message types supporting the major communication requirements for allocation of resources across the emergency incident life-cycle. It’s not one size fits all! Moreover, the Resource Messaging goes through three distinct phases of “discovery”, “ordering”, and “deployment”. The level of detail of the reference information would vary in each phase. Therefore, the message broker would need to manage those elements through out the life-cycle involving the resource messaging phases.

Is S3XML Versatile?

Relating to an experience, of my own, with an application involving EDXL-CAP and the use of Eden’s native S3XML for generating Common Alerting Protocol (CAP) compliant document outputs, what I realized in EDXL-CAP was it was only producing the set of data for the specific GUI and not the entire CAP message. For example, the segment of a CAP message had it’s own GUI to edit the specific values. Within the segment GUI, the S3XML output would only produce the XML for the Alert segment and not the rest. 

S3XML is not GUI-specific, not at all. By default a standard REST controller provides the same output for a non-interactive request as for an interactive one, which is a meaningful behavior, though. However, the Point-Of-Interest (POI) exporter in Eden’s OpenStreetMap (OSM), for example, combines multiple data resources into a single output document, and is still both RESTful and S3XML and “inline-transformable”.

OData provides a generic interface standard to query data repositories of all kinds, including schema introspection. Eden’s S3XML is very similar to OData, although it does not implement aggregation methods, yet. APIs with (semantic) schema introspection capabilities have a much higher potential to facilitate interoperability than emergency management specific data formats; especially with non-emergency-management specific applications. However, that may be an over statement, while such level of achievement is still a bit away. However, Humanitarian eXchange Language (HXL) is a big step into that direction.

EDXL-HAVE, Yet Another Experience

“One of the problems encountered with EDXL-HAVE, namely the standard data structure for Hospital Availability Exchange, is that it assumes the role of an “emergency manager”, a decision maker who controls the resources and is thus in need of the information – and that organisations are ready and open to provide it (i.e. a hub-spoke model).”

In the EDXL-HAVE standard, it is assumed that hospitals (or their operators) report to that decision maker(s) on their available capacity, and the decision maker responds to that information, though not by routing patients but by acquiring and deploying additional resources as needed. This doesn’t work where the response decision happens entirely decentralized. In multi-national scenarios, why would India even respond to Pakistan’s HAVE data needs, even if it was ready to assist in an event?

In such cases the information flow is more peer-to-peer and organisations are expected to self-organize. High-level decision making is based on aggregated information whereas details may not be shared at all. Of course, organisations can make pre-event agreements to share HAVE information – but not at an aggregation level of their choice! Using the protocol requires thus both organizational (to actually collect the data at the required level of detail) and political (to share the data) change upfront. It would be better if HAVE (as well as other EDXL standards) had an integrated aggregation pattern, so that you can easily choose the level of detail that fits for the particular case without having to adapt your application.

Does EDXL Require Organizational Change?

EDXL is perceived a domain model requiring organisational change upfront, instead of one that is easy to implement on existing models. The most common answer received from developers that Sahana has interacted with was “EDXL wasn’t designed for the cases I was talking about” To that end, we wonder what the common denominator for EDXL standards is? In most cases it is FEMA and the US way to deal with disasters. Although originating in US, EDXL-CAP is the only initiative that the US nor FEMA was the early adopter. Perhaps one reason to it gaining wider scale global adoption with several Nations and Alerting related Vendors implementing the EDXL-CAP standard.

A non-advocate of standards may see EDXL very much as a US-specific thing. In fact, he had no requests so far for any EDXL support outside the US, and especially not at the INGO level (e.g. United Nation Organizations). New, RDF-based (Resource Description Framework) approaches, like HXL seem more promising these days and may outperform all the use-case specific standards in the long run. It is hardly feasible (or even desirable) for most organisations to adapt their data flows to EDXL requirements. Hence, they prefer standards which adapt to their existing resources, like OData or S3XML, rather than the reverse practice. 

The philosophy of letting everybody do their own thing and yet be interoperable is simply more adequate than the idea of making everybody do the same in order to be interoperable. However, there is some level of consistency required with the emergency data exchanges. Relating the concept to disparate spoken languages – to foster a harmonized meaningful conversation requires an “interpreter” to manage the conversation between two people speaking in the two different languages. Human beings, to date, are far more intelligent than machines and are capable of processing with incomplete information to take on the role of an effective Interpreter. However, machines are relatively inept to allow them to do their own thing and yet expect them to interoperate without some level of coherence.


Interfaces can be semantically incompatible even when they implement the same syntactic EDXL standard. Still, there are cases of people interpreting EDXL elements differently in different contexts. Some argue that this is due to the lack of rigor in the standard. It may be a consequence of the top-down ontology approach in EDXL.

EDXL doesn’t force people to adopt them and say this is the bible you should practice the religion. However, what their intention is to provide a set of elements and a data structure that allows an implementer to think through to ensure their messaging is coherent to a certain extent. I think that’s why most of the elements are set as optional which allows the implementer to build their own policies around them; meaning work flows.


Anonymous said...

As a statement of use of EDXL:

1) we haven't had any deployment so far in which EDXL was effectively used for data exchange

2) apart from CAP, there are currently no ongoing development efforts regarding EDXL imeplementation in Sahana Eden – because we don’t currently have any use-cases where it would add value, or where it had been requested

3) we have had cases where EDXL was considered, but eventually dropped in favour of generic data formats and direct end-to-end API requests

4) main reasons to my experience:

a) the overheads for EDXL message tracking/brokering while not adding value to the particular case we had – not generalized like it appears in your blog entry here

b) the coexistence of other simpler more common interface options that can be generalized

c) the complexity of EDXL vs. desired simplicity of use-cases

d) divergent ontologies resp. data/process models that could not be modified without organisational change, or get mapped with reasonable effort

5) so far, we’ve not seen any demand for EDXL outside of the U.S. except for CAP (meaning: they may exist, but we haven’t seen them, may thus be specific for Sahana Eden and the domains it’s typically used in these days)

EDXL-RM might have been an appropriate solution, for the client, (I still think it is). however it was a scope decision made by the project manager considering development costs vs. benefit.

In hindsight, I think that the additional costs for EDXL message brokering and the co-existence of a simpler, generic solution for the required data exchange (S3XML in this case) might have been the key reasons for this decision.

By no means does that mean that S3XML is an alternative for EDXL – let alone a practical choice!

Generally, S3XML is an internal format of our API, and not designed for data exchange with other systems. It should not be advertised for that purpose anywhere because that would obviously undermine our flexibility to adapt the format to our API requirements. For data exchange with the rest of the world, it is a fundamental principle – and the main strength – of the Eden API that it can adapt to the formats needed/provided by peer applications. This is achieved by its architecture consisting of pluggable API adapters, format codecs and XSLT transformation stylesheets.

My personal standpoint is that this should be much less focussed on data /formats/, and much more about common ontologies. Using a common syntax alone is good to exchange *data*, but rarely sufficient to exchange *information* for which you also need standardized or at least commonly agreed semantics.

I hope I have made myself a bit clearer now.

Nuwan T. Waidyanatha said...

Absolutely agree with you that adoption of standards should consider the economics with the cost and benefits of the adoption.

As mentioned if the Sahana Eden or client tools do not intend to interoperate, then yes EDXL is not a must.