By Rich Wolski | April 09, 2012
With the agreement between Eucalyptus and Amazon to cooperate on ensuring API fidelity between the two technologies comes the question of defining what it means for two independent implementations to support the same API. The API itself consists of a large number of different "commands" or "service requests," each of which triggers many different operations within the system that are visible to the user. Moreover, the API makes specific assumptions about "things" (VMs, storage volumes, network addresses, etc.) it is defining and manipulating. Launching two different VMs with different network and storage characteristics through the identical API call doesn't capture fidelity.
Thus, API fidelity measures three dimensions of compatibility in the interface "calls" it implements:
- the calls and parameterizations of each call
- the actionable responses generated by each call
- the similarity between the objects manipulated by each call
Note that it is not enough merely to count the number of calls and/or to look at the parameterizations to determine compatibility.
For example, almost all cloud APIs include a simple call that allows a user to launch a virtual machine (VM). For two implementations to implement this API call, the call must be present and must parse the same set of parameters, it must also inform the user of the status of all activities associated with the launch (address assignment, identity, etc.) in the same way and in the same order, and the VM that is launched must be the same VM (same access method, the same device characteristics, the same composition, etc.)
It is also necessary to look at the interaction between API calls, particularly in a cloud setting where the access control for separate resources interact. For example, running VMs can refer to infrastructure resources that have been created and/or allocated by an end-user. If the user deletes a storage volume or de-allocates a network address while the VM is using it, does the VM lose access? Does it terminate? Or does the cloud simply refuse to obey the user because the VM is using those resources? Note that all possible answers are rational, depending on whether the resource is considered to be controlled by the user or the VM to which it is allocated. Thus, in addition to replicating the specific semantics of each call and cloud object, it is also necessary to consider the way in which the API calls interact when engineering API fidelity.
For these reasons, ensuring API fidelity is often a long or even never-ending process. It is tempting to implement a set of calls that appears to have the right arguments and responses, but doing so is simply the first (and often easiest) step in the implementation.
Note also that the majority of the valuable fidelity is embodied in a small subset of the API for many use cases. For example, the original University implementation of Eucalyptus was able to support a large-scale weather forecasting application running simultaneously in several data centers and in AWS using about 15 of the API calls. The fidelity of those calls had to be excellent but with that fidelity came the ability to run a legacy applications with code segments that were more than 30 years old.
Eucalyptus and Amazon are now working together to support and extend the use cases for which API fidelity makes this kind of success commonplace. Which explains, at some level, why we decided to make Eucalyptus open source and to work so hard on solving the tough technical problems that come from making cross-platform execution possible. We believe that the users and customers of Eucalyptus benefit from a fundamentally new way of using infrastructure when we make all infrastructure components -- public cloud and on premise -- work as one easy-to-understand and easy-to use tool. The original University version of Eucalyptus showed us what would be possible and we are excited to be bringing this vision to fruition. We welcome your participation in achieving the best possible solutions to the problems that stand in the way.