DevOps is a software engineering methodology that attempts to combine the disciplines of software development and system operations into a unified practice. Prior to the emergence of DevOps, software development was a highly segmented undertaking. Programmers made the code and then gave it to Quality Assurance (QA) to test. If the code passed QA, it was passed onto the Operations department to deploy out to users. Code that failed QA was sent back to the programmers. Programmers had little insight into the activities of QA and vice versa. And, neither group really understood the activities of Operations. Everybody lived in their own silo unaware of the activity of others. The process made development slow. The code a programmer sent out in September might not make it to Operations until October, if not later. Long release cycles deprived companies of the time to market advantages they desperately needed to compete in business.

Long Release Cycles Affect the Entire Industry

However, long release cycles are not just confined to programming, QA and operations. They’ve plagued computing since early UNIVACs started collecting census data back in 1951. The history of software development is one of laborious, time consuming activities, very similar to the automobile industry prior to the advent of the assembly line. Just as each car was hand assembled, so too was each piece of software that made its way from design to eventual release. Code passed through a lot of hands before it made it out the door to users.

In the beginning, before computers became a consumer commodity, the industry could tolerate long release cycles - recall that early customers for computers were big companies and government. Waiting years for a product was the norm.

The general software development practice at the time - both internal in-house and third party commercial - was based on the waterfall model. The architects designed the product; programmers coded it; QA tested it and Release Management deployed the programs to Operations which made the applications available to end users. Any failure along the way usually meant that the code went back to the starting point far upstream.

However, once PCs hit the scene, the demand for software increased dramatically. Everybody wanted code - military, businesses, schools, universities and consumers. Not only did this mean that a software company would need to keep revising its software to meet all the demand, it had to create new software for previously unserved markets as well as new emerging markets. Remember, Microsoft’s path to prosperity did not stop with the BASIC programming language. The company went into operating systems and then Office, SQL Server and entertainment software such as Flight Simulator.

Designing, creating and marketing a single product, such as a programming language, is hard. Manufacturing tens, if not hundreds of products, is a herculean, time consuming, effort. Despite technical advances, release cycles were still long. Windows NT 3.1, the first viable version of Microsoft's 32 bit, graphical operating system was released in 1993. The more stable Windows NT 4.0 was released three years later. And, not only were release cycles long, sometimes the products that came out as a result were multi-million dollar duds that few wanted, such as: MS-BOB.

Microsoft was not alone facing crushing demand that was becoming harder to meet. The whole industry had the problem. The waterfall model was failing. Something had to change. And it did. Along came Agile. 

Agile Arrives

In February of 2001, a group of the most influential developers, system architects, product managers and project managers in the software industry convened in a ski resort in Utah for two days in order to come up with a better way to make software. The result of that meeting was the Agile Manifesto, which is as follows:

The short list of principles defined are the essence of Agile. Agile focuses on promoting a high degree of collaboration among and between self-organizing and cross-functional teams, as well as the customers that will be using the products being made. Agile promotes short development cycles in which one or two features are released at a time. Short development cycles allow for faster testing, feedback and subsequent revision.

In terms of creating new products, Agile advocates taking the approach of developing the Minimal Viable Product (MVP). An MVP has the smallest feature set possible. A small feature set then serves as the baseline from which a more complex application evolves. The dynamic is similar to that of starting from a simple text editor and evolving into a multifaceted word processor. Instead of focusing on a Big Bang release, teams focus on defining, coding, testing and releasing features individually, rather than the grander vision all at once.

Agile has gained acceptance not only in the software development community, but also among other industries that make products and services focused on meeting the needs of customers. As Agile has become more popular, the methodology has become more formalized to support its practice. Most project management tools support Agile based concepts such as user stories, scrum, backlog grooming and burndown rate. Agile has become the norm rather than the exception for software development.

Virtualization Gives Rise To DevOps

DevOps is a direct descendant of Agile. DevOps builds upon the foundation Agile established. Just as the rise of the personal computer created the demand which instigated the birth of Agile, machine virtualization made possible the continuous integration/continuous delivery (CI/CD) principles of DevOps.

Machine virtualization makes it so software can emulate any number of virtual computers per physical computer. More than that, each virtual computer can be provisioned for any particular need. For example, instead of hosting a database server on the same physical machine as an FTP server, they can be hosted on their own virtual computers, or, virtual machines (VM) provisioned specifically for their own individual needs. And, since VM is software, this means that any computer can be created, on demand, by just a script. Virtualization makes possible the DevOps principles of infrastructure as code and automation everywhere. You can’t code a physical router into existence, but using Software Defined Networking, you can. Not only can you code the router into existence, but you can also write the automated test that will exercise that router once created. This brings us to the promise of DevOps.

In the past, programmers talked the language of application development, QA talked testing and operations personnel talked provisioning and deployment. The words used might be the same, but the meanings were different. Communication between groups was hard. Today virtualization, automation and the coding practices that go with it all have become the lingua franca among all members of the software development team. There may be different dialects, but the language is essentially the same. Communication becomes easier. Faster, clearer communication speeds up release cycles. Also, automation at all levels - from unit testing, to code deployment, to large scale system testing, to release provisioning and deployment - provides the reliability and ease of replication that makes release cycles even faster. If something goes wrong in a load test, upstream personnel don’t need to spend days figuring out how to replicate the problem. They can simply run the scripts that provision the environment and test the problematic code. These same scripts were are available to everyone, so anyone can code and test without reinventing the same wheel throughout the production cycle.

So, to go back to the original questions, what is DevOps and how is it different from Agile? The short answer is, DevOps is the practice of combining the best practices of development and operations into a single, unified discipline. Agile is the general set of principles from which DevOps evolved. Both are important. Agile and DevOps make it so that more high quality software gets released to end users faster. And, as those of us who have dedicated our lives to the profession understand, making software that counts is what really matters. Agile and DevOps provide the guiding principles that allow us to fulfill our mission.