By Rich Wolski | June 27, 2012
The release of Eucalyptus 3.1 marks the end of the beginning for Eucalyptus 3. In February of this year, we released Eucalyptus 3.0 -- a fairly extensive re-factorization of the Eucalyptus platform to support features, such as high availability, identity management, and bootable storage volumes, all of which required intrinsic changes to the implementation of the cloud fabric itself. We also needed to change the way Eucalyptus is modularized to allow easier more "pluggable" deployments and smoother integration and deployment for Linux. Eucalyptus 3.1 completes the restructuring of the platform (begun with 3.0) to meet these requirements.
Much of the focus in the 3.1 release has been on making Eucalyptus more accessible as an open source platform. In particular, Eucalyptus 3.1 introduces:
- A new plug-in architecture and package-driven modularity to make automated installation and upgrade easier
- Better dependency management bringing Eucalyptus into compliance with Linux distribution rules
- New community-facing projects and services such as GitHub hosting for the source base, Eustore, Eutester, and Engage
- Heavy-duty Quality Engineering in the form of large-scale automated testing coverage
I'm excited about the evolution of Eucalyptus, both in terms of its capabilities and its community, and I will try and highlight some of the noteworthy features of the 3.1 release in this posting.
The original design relied on programming language-specific features to define and enforce architectural modularity. For example, pluggable modules took the form of dynamically loadable routines that could supersede existing functions. Often, however, a plug-in would need to interact with routines that span Linux packaging boundaries.
Release 3.1 is designed so that the plug-in architecture takes into account both plug-ins and package boundaries. It is now possible to develop and deploy plug-ins as separate packages rather than as code that touches multiple packages in a Linux deployment.
The benefit of this re-factorization is that it permits the extension of Eucalyptus from an existing package-based installation without modification to the un-extended code. The code for interfacing Eucalyptus to various Storage Area Network (SAN) devices, for example, can be packaged and deployed separately from the core of the platform. Users and customers who have Eucalyptus deployed in a production setting will be able to plug and unplug modules based using the new packaging approach. At the same time, Eucalyptus developers can now use the new modularity to develop their own extensions in a way that allows them to be packaged separately.
Linux Distribution Friendly
Eucalyptus 3.1 also includes new dependencies that make it possible for Eucalyptus to conform to the distribution rules set forth by Debian, Ubuntu, and Fedora. Eucalyptus 1.0 and 2.0 were both "distro compliant" with respect to their dependencies. Several of the new features introduced at release 3.0 necessitated new dependency relationships that that we could not fully align with various Linux distribution rules. We have fixed these inconsistencies at release 3.1. To make Linux distribution generally possible in the future (hopefully as soon as Release 3.2), the 3.1 code base also now depends on open source software packages in a way that meets the requirements for inclusion in the various Linux distributions.
In addition, we have packaged and are distributing Eucalyptus 3.1 for Centos 5 and 6, RHEL 5 and 6, Ubuntu 10.04, and Ubuntu 12.04. Combined with the new packaging and plug-in architecture, release 3.1 is the most Linux friendly Eucalyptus release to date.
Easier Access to Code
With the re-factorizations in Eucalyptus 3.1 came the opportunity to update the source code repository to make the code base more accessible. The Eucalyptus code repository is now available on GitHub and Eucalyptus Systems has adopted git to offer better source code access to the developer community making contribution easier and more straight forward.
Using GitHub and git accelerates the pace of Eucalyptus community projects such as Eustore (a new image repository effort) and Eutester (a community testing and debugging project). Looking forward, we believe that the move to git and GitHub will best serve the needs of the Eucalyptus open source community with respect to building out the code base.
From HA to QA and Back Again
Perhaps the least appreciated but most important innovation in Eucalyptus 3 is the development of the automated QA system and its application to the release process. High Availability (discussed at length here and in a sequel here) in Eucalyptus is really a combination of a new architecture and a new, fully automated QA system for testing and verifying its function. The 3.1 release is the most heavily tested version of Eucalyptus so far. When completed, the QA system will have performed 500,000 individual tests, launching more than 450,000 instances, across 6 Linux distributions, on 3 hypervisors, using 4 different storage architectures.
Eucalyptus 3 as Production Infrastructure
Personally, I'm thrilled to see 3.1. Our goal, when we transitioned from a research project at UCSB in the Computer Science Department was to help users, developers, and customers to define and then build an open source cloud platform that could serve as critical software infrastructure for the enterprise. It is both fun and exciting to see the third generation of Eucalyptus (begun a little more than 18 months ago as a collaborative developer-customer-user effort) come to fruition.
As Marten Mickos (our CEO) intimates in his blog on the Architecture of Participation, software is but a small part of what makes up the Eucalyptus community architecture. In addition to GitHub, Eustore, and Eutester mentioned previously, customer and community support for Eucalyptus are unifying with the 3.1 release. Eucalyptus Engage pulls together issue and bug tracking, community discussions, and the ever-growing knowledge base into a single, open venue that will be hosted and moderated by Eucalyptus support. Engage is a great step forward and we are excited about the future of community participation in helping our company grow.