Tech Perspective: A Look Inside the AWS-Eucalyptus Agreement

By Rich Wolski | April 13, 2012

In March, we announced an agreement to collaborate with Amazon Web Services (AWS) to ensure API compatibility between AWS and the Eucalyptus open source on-premise Infrastructure as a Service (IaaS) platform. Beyond the details of the news, I'd like to shed some light on how I think this agreement helps customers focus on the larger discussion of the policies and business logic that govern application workloads.

For some time now, Eucalyptus users have implemented application deployments using a combination of Amazon AWS and locally owned resources. Indeed, the original use case for Eucalyptus was to support large-scale science applications that could draw resources simultaneously from AWS and a geographically distributed set of data centers, each operated by a separate organization. From the beginning, the implementation of a consistent set of abstractions for using AWS and on-premise resources, accessed through a single API, was a goal for Eucalyptus.

Based on this and other early experiences with on- and off-premise distributed computing, the Eucalyptus community has advocated that interoperability at the API level between Eucalyptus and AWS is an essential feature for the Eucalyptus platform. AWS is, by far, the most popular and successful cloud computing environment available today. Amazon Web Services provides a highly reliable, scalable, low-cost infrastructure platform in the cloud that powers hundreds of thousands of businesses in 190 countries around the world. The agreement between AWS and Eucalyptus enables better interoperability between AWS and on-premise IaaS with Eucalyptus.

Given the announcement, what will Eucalyptus truly enable?

In other words, "What does a consistent view of infrastructure, regardless of premise (i.e. premise consistency) provide that API compatibility does not?" I believe the answer to this question is "policy elasticity," or the ability to exploit elasticity on behalf of a workload based on policy rather than technology. API compatibility is certainly necessary. For workloads to leverage both cloud computing and internal infrastructure resources as a consistent platform, they must be able to interact with the platform in the same way regardless of where they are hosted – on-premise or off. However, because the benefits of cloud computing and on-premise IaaS environments are complementary, API compatibility enables elasticity to be a matter of policy and not mechanism.

It is the raising of the level of abstraction — from mechanism ("plumbing") to policy ("architecture") that I find to be the exciting possibility enabled by an agreement between Eucalyptus and AWS.

With the interoperability mechanism no longer in question, we can begin to discuss IaaS in terms of the policies that govern application workloads and not the technological boundaries between infrastructure domains. Indeed, business logic (often encoded in an application ensemble and the data over which it operates) is better expressed in terms of policy than in terms of technology. Raising the level of cloud abstraction above the API level moves the cloud closer to the application and, ultimately, to the business logic governing the application.

What does this agreement on API fidelity mean for open source technology?

Note that the ability to architect infrastructure from the perspective of policy and not mechanism initiates a new field of innovation with many immediate challenges. At first, for example, premise consistent deployments will be built and administered manually, much as they are today. With Eucalyptus' commitment to maintain API fidelity with AWS however, comes the opportunity to develop new technologies focused specifically on cloud governance.

Because Eucalyptus is open source it will continue to serve as a platform for community innovation, but with the addition of support from AWS for Eucalyptus to stay consistent with its APIs. This announcement enables the open source community to focus on new models of governance. As James Urquhart quipped recently, "Cloud computing is about agility governance." Combining cloud computing and internal infrastructure through a consistent set of abstractions certainly presents new opportunities for agility. However, these opportunities portend new technical challenges associated with implementing "governance." The open source community and its diversity of goals and requirements for technology will undoubtedly be the engine of innovation in this area.

Looking Ahead: Elastic Policies

I believe we will see the invention of policy elasticity. Today, the term "elasticity" is almost definitional with respect to cloud computing. Infrastructure is not "cloudified" if it does not support an "elastic" infrastructure model -- one in which application resource usage is expressed strictly in terms of capacity and not specific infrastructure components or technology.

The insurance of compatibility with the AWS APIs, however, brings the possibility of defining similar flexibility at the policy level. It should be possible to express "how" applications access resource capacity and not "what" infrastructure (cloud or on-premise) they use. Put another way, I believe we will see a future where cloud application deployments are expressed in terms of policies governing resource usage and not identifiers specifying which cloud resources should be employed. Working at this level of abstraction further separates the application (and indeed business logic) lifecycle from the technology and infrastructure lifecycle. It is this separation of concerns that enables greater agility.

Personally, it is delightful to see Eucalyptus work more closely with AWS. Designed originally for use with AWS as a platform for large-scale science applications, it is now seeing rapid uptake in an amazing variety of contexts in the way we, the University researchers of Eucalyptus, did not anticipate. Developers of the leading gaming platforms, companies fielding innovative large-scale telecom applications, technologically advanced government agencies, and the most global corporations are all turning to Eucalyptus to help them realize a consistent view of their infrastructure. Amazon gives them highly reliable and scalable cloud computing infrastructure, and Eucalyptus gives them the robust on-premise IaaS platform. Together these capabilities form a powerful substrate for the next generation of scalable application deployment. I am looking forward to understanding this next phase in the development of cloud computing and how the agreement between AWS and Eucalyptus will help enable its evolution.

Get Started with Eucalyptus

Use FastStart to easily deploy a private cloud on your own machine from a single command!