Achieving Agility in Dev and Test: Securing the Beachhead for Your Cloud Journey

Preparing for a cloud transformation can feel like the view of Everest from basecamp: How do you initiate a journey of such magnitude? Of course, the answer is preparation and a graduated plan of attack that advances one step at a time. But, specifically, where do you start?

When it comes to cloud projects, despite the early days, one pattern has become clear for IT: You start with a private cloud, looking to development and test functions as your initial use cases.

But when you do, you also must be mindful of the hill—or mountain—you're looking to climb. Your earliest decisions, while not insurmountable, can substantially impact the end result.

That's why it's important to enter your early dev/test forays with a clear view of your full lifecycle aspirations. That's the purpose of this paper—to help you define and design your perfect dev/test cloud with an eye to the future, making decisions that will deliver near-term benefits while setting you up for long-term success.

In order to understand what it takes to deliver a dev/test cloud, let's first explore the organizational dynamics we need to accommodate and overcome.

The Dev/Ops Divide

There's a classic divide between software development and IT operations groups that transcends one- dimensional stereotypes of incompatible personalities. It's not the case of "dev is from Mars and ops is from Venus" as much as it's the result of different priorities and institutionalized processes that stand in the way of effective collaboration and truly high-performance IT.

The truth is that development organizations are driven by speed, while IT is driven by control.

Diagram of Dev Ops Speed and Control

When these two forces work in conflict, it creates division and dysfunction, which is the current state of many IT organizations. However, when they work in concert they enable agility—the very goal of cloud computing.

Today, the rift between dev and ops is so pervasive that it has given rise to an industry-wide movement— DevOps—that explicitly targets the bottleneck between these groups. Perhaps more notably, it's this dev/ops divide that has catalyzed the growth of cloud computing as a megatrend.

Why Wait Weeks When Minutes Will Do?

The rise of public cloud has demonstrated the power of on-demand IT. It used to take weeks or months to get infrastructure or software provisioned. Now, with the swipe of a credit card, developers and business users have what they need. Why didn't we think of this before?

The public cloud model is shining a light on corporate IT, begging the question: If they can do it, why can't you? It's forcing a transformation in the design of IT delivery models from ticket-driven provisioning to on-demand self-service IT.

That's the important role of private cloud—providing the abstracted service interfaces, standardized offerings and automation for fulfilling demand at the speed of cloud.

The reality is that this sort of responsiveness is exactly what development and testing organizations need to do their jobs effectively. Today, it can take days or weeks (or longer!) to provision the hardware developers and QA engineers need to get their work done.

This leads to time to market delays that impact top- and bottom-line business performance; and lost productivity, as engineers are forced to wait on the sidelines for infrastructure to arrive In many cases, it also leads to a perverse side-effect of provisioning delays, where engineers hoard capacity, squirrelling it away out of fear that they'll never get it back! Such hoarding not only impacts resource utilization; it often leads to system inconsistencies caused by software and configurations that drift from their original definitions, leading to support and compliance issues.

All of these scenarios are testimony to the current state of so many enterprise IT organizations. The rise of public clouds have provided an escape path for unfulfilled demand, offering a speedier alternative for developers and QA—but often at the expense of the control operations teams require.

The Rise of the Private Cloud

Your developers may have had some experience with public clouds, but chances are, you're corporate IT strategy begins inside the firewall. It's the need for control that has led enterprises to private clouds as the starting point on their cloud journey. While the long-term goal is often a blend of both private and public clouds to form a hybrid cloud fabric, private clouds are often the beachhead to the cloud transformation. Why?

  • Policy—regulatory and corporate compliance restrictions around security and data privacy mean that some applications will never see the light of day outside of the corporate data center. These policies include PCI for companies that handle credit card data, HIPPA for those handling medical records, and all manner of other public and private sector regulations. For applications subject to these policies, public cloud may be a nonstarter, even for dev and test.
  • Cost—most enterprises have made substantial investments in data center infrastructure, which is a sunk cost on the balance sheet that they would prefer to utilize as part of their cloud initiatives. When you have racks of servers available in the data center, it simply makes better financial sense to start there.
  • Control—private clouds allow IT organizations to define and enforce the control and governance mechanisms that make a cloud environment efficient, compliant and integrated into an existing ecosystem of tools and processes. This may include integrating workflows, identity management, approval processes, access controls, usage and quota metering, chargeback, showback, etc.
  • Performance—simply put, it's easier to ensure predictable performance when you own the infrastructure. Public cloud performance can be non-deterministic, which makes is harder to objectively assess the performance characteristics of an application because its behavior isn't easily isolated from its environment. Particularly for I/O intensive applications, private clouds often deliver lower network latency and more predictable, discernable run-time performance.
  • Abstraction—tuning and optimizing applications often requires "lifting the hood" to tweak underlying infrastructure. While abstraction is a key enabler and important advantage of both public and private clouds, only private clouds allow you to access infrastructure below the abstraction layer for those moments when you need to lift the hood. As a result, when enterprises set down the cloud path, they often begin with private clouds. And when they begin with private clouds, they often get started with dev and test use cases.

Why Cloud is the Right Architecture for Dev and Test

It's become almost axiomatic to suggest dev/test as the starting point for a cloud initiative. Why? Because the dev/test function lines up perfectly to cloud architectures. Specifically, cloud:

  • Matches the need for disposable infrastructure—dev/test demand patterns are unpredictable: without a cloud, the 50 servers required this week may sit idle next week. Cloud transforms infrastructure into an elastic utility that can be spun up and spun down as necessary, making shared resources available on demand for faster application provisioning and the sort of infrastructure scale required for load and stress testing.
  • Solves the dev/ops riddle—making infrastructure available on demand to improve utilization and productivity of engineering capacity and to accelerate application delivery.
  • Extends agility downstream—many dev and test organizations have implemented Agile and other iterative practices that help accelerate delivery of working application functionality. But these investments are often constrained by provisioning steps, which remain anything but agile. With a cloud strategy, agility is extended across the application development lifecycle.
  • Enables repatriation of cloud apps—once an application has been designed to run in a public cloud, it's often hard to redeploy behind the firewall. However, over time, many IT organizations want to repatriate these applications to optimize costs and to maximize control, particularly when applications grow in scale and scope of usage. With a hybrid-optimized private cloud, these applications can be easily "brought home."
  • Supports the continuous innovation mandate—allowing dev/test organizations to experiment with rapid prototypes, turning creative instincts into new application breakthroughs or "failing fast" by invalidating prototypes early in the cycle. Cloud allows prototypes to be rapidly deployed to targeted user segments for feedback through A/B testing and other in-market data that ensures delivered applications are well aligned to user requirements.

Building the Ideal Dev/Test Cloud

Cloud can dramatically improve the cost-efficiency and responsiveness of your IT processes, but—let's face it—it's an architectural transformation that requires foresight, preparation and planning. Like any other transformation, your cloud project should begin with a plan—and that plan must begin with the end in mind.

You may be in the dawn of your cloud implementation, but the decisions you make today will impact the long-term success of your project. For example, many organizations set a vision for a hybrid cloud future where workloads can move fluidly between public and private deployment environments based on optimizations for price, policy and performance. It's a beautiful future because it enables openness and choice, commoditizing infrastructure and allowing IT to drive down cost and to better align infrastructure resources to business requirements.

Over time, you may want to allow unexpected demand to spill over into the public cloud by "cloud bursting," or you may see the public cloud as a solution for disaster recovery as part of your business continuity strategy. Or perhaps you're considering a cloud strategy that blends private and public across lifecycle stages, where each stage has a different primary and secondary deployment environment.

To enable any of these and other hybrid use cases, you need to begin with an eye toward deep compatibility, where the APIs and semantics of your private and public clouds are near-perfect mirror images. The consequence to not getting this right up front is substantial delay in achieving workload portability between environments as you evolve your cloud to become a hybrid fabric.

Picking the Right Architecture

In the early days of a market, every decision can feel like a high-stakes wager. In the absence of historical data, it can be a challenge to know where to lay your chips on the table.

Fortunately, as it relates to selecting the right design pattern for your cloud, Amazon Web Services (AWS) stands alone as the overwhelmingly dominant architecture in use today. Some analysts estimate that Amazon commands more than 70% of the public cloud market today.

Because of its dominance and demonstrated ability to rapidly innovate, Amazon has quickly secured a nearly unrivaled position as the de facto standard for cloud computing.

So, as it relates to a placing bet on the architecture for your cloud initiative, the smart money is on Amazon as the best framework for your private cloud. Amazon is more than a cloud vendor; it's a massive and growing ecosystem that reaches across developer communities, commercial and open source software and the tooling and automation that enables clouds.

Betting on a private cloud that closely adheres to the Amazon APIs and aligns to its semantics will pay dividends by enabling hybrid use cases that allow you to dynamically shift workloads between public and private environments for cost, performance, policy and availability.

Anatomy of a Dev/Test Cloud

With an established vision for your dev/test cloud, the next step is to get focused on the bill of materials. But before we dive into the piece-parts that comprise your dev/test cloud, it's important to first discuss the difference between a cloud environment and self-service VM provisioning.

Too often, our understanding of technology is muddled by vendor-spun attempts to co-opt a category. There's no small measure of "cloud washing" taking place today as wholly unrelated or tangentially related vendors attempt to attach to what they see as a hot spending pattern.

Ask yourself: What makes a cloud a cloud?

The reality is that simply slinging around VMs ever faster through automation may very well improve your application release cycles, but it alone is not a cloud.

A cloud provides a framework for building and deploying applications that are inherently elastic in nature— they can scale and descale dynamically based on changing demand patterns. A cloud is ideal for modern applications subject to dynamic usage driven by the web, social and mobile and other factors that impact virality and unpredictable scale.

It's important to understand that, while a end-user portal of pooled VM resources can certainly improve the efficiency of your IT processes, this is not a substitute for a cloud. With that caveat, let's review the attributes of your dev/test cloud:

  • Infrastructure as a Service (IaaS)—provides the automation for dynamically provisioning and de-provisioning compute, network, security and storage resources as application and workload demand ebbs and flows. Think of IaaS as the automation and abstraction that turns raw infrastructure into elastic resources that can be provisioned and scaled on demand and shared across applications. As a foundation for your cloud, it's important to select an IaaS platform that aligns to your hybrid cloud vision.
  • End-user Portal—provides a user-facing portal for publishing and consuming standardized infrastructure, platform and application offerings. The end-user portal is designed to remove the bottleneck of traditional provisioning processes, allowing users to discover, evaluate, request and provision resources, in minutes, without IT intervention.
  • Governance—allows IT to set and enforce policies and workflows to enable efficient automation of provisioning processes. These policies often include approval steps, usage quotas, service level agreements, pricing and internal accounting procedures to ensure the chargeback or showback that helps to throttle demand. It's important to understand the significance of this last part: without an accounting model in place, users have no incentive to economize the usage of resources. An accounting model helps to create an inefficient marketplace where providers and consumers meet around supply/ demand equilibrium.
  • Environment management—applications run on operating system and middleware platforms, not raw infrastructure. Therefore, it's important to have automation for rapidly provisioning these environments on top of your cloud infrastructure. This includes templates that describe the content of these environments, version control for maintaining the lifecycle of these templates and the ability to generate images from these templates for rapidly provisioning and configuring environments and making necessary adjustments at run time.
  • Integration with canonical repositories—which provides the straight-through integration back into the dev and test chains for continuous deployment and change of application and middleware. Think of this as your automated supply chain where integration to source control management systems (SCCM), build automation or continuous integration tools is your connection back to the sources of record. When changes are made, they can be seamlessly propagated through the cycle, enabling the speed and agility of continuous delivery.

Considerations on the Path to Cloud

We're now in the waning hours of the early days of this cloud transformation, which means that—while we don't have all the answers—some truths are beginning to form as best practices. Here are a few things to consider as you begin your cloud journey:

  • Secure executive sponsorship—always look to solicit active and visible support from someone in the executive ranks to act as a sponsor and advocate for your project. They'll help get it off the ground and will bring visibility to your early successes to create momentum.
  • Start small—don't boil the ocean with your initial cloud project. Instead, target a relatively small, technical user population and make them part of the project. They should feel a vested interest in the success of this initiative, providing constructive feedback and working collaboratively to iteratively improve the experience provided by the cloud. This makes software developers and QA engineers ideal initial user communities for your cloud.
  • Begin with the end in mind—think strategically about the growth path for your cloud and the use cases you want to enable. Don't waste this initial effort by designing it to be disposable. You should use this early experience as a safe environment for refining the definition of your broader cloud strategy and the architecture, processes and tool chain that enables it.
  • Create a microcosm—think of your dev/test cloud, not as a hobbled version of your full-blown desired- state cloud, but as a smaller-scale incarnation of the same thing. Don't work in half-measures simply because of the small user community or narrow use cases. Instead, integrate and exercise the entire automation tool chain, gaining valuable experience that will pay dividends when it's time to scale up and out.
  • Design-in flexibility—select an architecture that enables compatibility with public clouds and other private cloud environments. Remember: This is the first step toward a hybrid cloud fabric that enables fluid portability of workloads between environments. That's the long-term win. Be sure today's thinking is aligned to that eventuality, despite the distance of its horizon.
  • Merchandise success—don't forget to take credit for your success, building visibility with internal stakeholders to galvanize commitment for expansion over time. Success begets success, creating the political capital and goodwill you'll need to take this to the next level.

Conclusion: A Cloud is Born

Congratulations! You've got yourself a cloud. Or, at the very least, you have the basic building blocks achieving your goals. Remember: Start with private cloud, plan for a hybrid future and begin with dev/test as the beachhead for your cloud journey. Contact Eucalyptus to learn how we can help you connect the dots from vision to reality. With more than 25,000 clouds started under its belt, no vendor is better qualified to be your partner in the cloud transformation.

Get Started with a Eucalyptus AWS-compatible Private Cloud

Try Eucalyptus for free on our machines or yours with our hosted Eucalyptus Community Cloud (ECC) or by downloading our FastStart automated installer.