DevOps is an architectural pattern, and architecture is a reflection of the organization.
Those familiar with 1990s American politics have likely heard the phrase "it's the economy, stupid." It was essentially the Clinton campaign's answer to the question of what single issue ultimately mattered to swing voters and consequently became its key message and unofficial slogan.
Although cliché, I feel the variant, "It's the organization, stupid," is a surprsingly fitting answer for some debatable yet important topics in the development world.
For this article I'm going to look at this answer's applicability to the question of "How do we ____ DevOps?" I never know which verb is appropriate--"do", "achieve", "implement", "enable", "become"--so pick one you're comfortable with; I'll be using the verb "blank" hereafter.
To answer this, let's begin by answering the more fundamental question of "What the blank is DevOps?" (that 'blank' is a noun). Here's a sampling of definitions from vendors in the DevOps space:
- NewRelic: "DevOps is a combination of software development and operations--and as its name suggests, it's a melding of these two disciplines in order to emphasize communication, collaboration, and cohesion between the traditionally separate developer and IT operations teams."
- Amazon AWS: "DevOps is the combination of cultural philosophies, practices, and tools that increases an organization's ability to deliver applications and services at high velocity... Under a DevOps model, development and operations teams are no longer 'siloed.'"
- Atlassian: "DevOps is a partnership between software development and IT operations that emphasizes communication, collaboration, and integration."
- Gartner: "DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams."
- Microsoft VisualStudio: "DevOps is the union of people, process, and products to enable continuous delivery of value to our end users. The contraction of 'Dev' and 'Ops' refers to replacing siloed Development and Operations to create multidisciplinary teams that now work together with shared and efficient practices and tools. "
All of these definitions more or less describe DevOps as the merging of two concerns/specializations/responsibilities commonly separated (or 'siloed') within IT organizations: development and operations. There are many other variants out there, but the consensus seems to be that DevOps (as the name itself suggests) is primarily a particular IT organizational phenomenon.
Say we want to play a baseball game. Certainly we would need bats, gloves, and balls (products). We would need to establish a batting line up, assign fielding positions, and perhaps create some practice regimens (processes). But even with top-of-the-line gear and the management services of Tommy Lasorda himself, we cannot play a legitimate baseball game with just, say, five individuals (people), no matter how skilled they may be.
The foundation for playing a baseball game is specifically eighteen people organized into two teams of nine, each having a shared set of responsibilities and a collective set of core skills. That's baseball's organizational phenomenon, if you will. Without such an organization, there is no baseball game. Similarly, if there's no such organization of individuals with development and operational skills (among others), collectively responsible and accountable for the products they develop and operate, I don't see any blanking of DevOps happening.
Nevertheless, vendors typically follow their DevOps defintion with a sales pitch for products or services. Microsoft and Amazon, in particular, include in their defintions of DevOps the necessity of processes/practices and products/tools in unifying development and operations groups. Maybe. But if DevOps is fundamentally an organizational matter, isn't the purchasing of products/services a non-sequitur? Or at the least, are not products/services a secondary concern for blanking DevOps?
Returning to the original question: How do we blank DevOps? It's the organization, stupid.
For those acquainted with the wisened "people, process, tools" triangle, you may be thinking, "Obviously, 'people' refers to the organization." Perhaps.
- "People" may be incorrectly perceived as being a matter of getting the individuals within the current orgnaizational structure "on board" with the DevOps movement. However, we know that our small group of three or four persons could be in complete agreement that they need to play a game of baseball and still, of course, be unable to actually play a game.
- "People" may also get misinterpreted as being a matter of hiring the right individuals into the organization--either employees or consultants. Either way, as mentioned before, Tommy Lasorda cannot help three or four individuals play a game of baseball.
- A final misperception of "people" is that DevOps transformation can take place by having upper management encourage the people (Dev and Ops groups) to "work more closely together."
Whenever I hear that, I immeidately think of exponentially increasing the number of communication channels in an already crowded situation.
While reviewing code recently, I noticed a similar design anti-pattern in several codebases developed by different, isolated teams: a multi-module Maven project with each child module representing a logical layer of the application architecture.
All of the modules had the same basic structure: a package of POJOs, a package of interfaces, and a package of interface implemenations, mindlessly named "{InterfaceName}Impl" (except for one disturbing case where the interface's name ended with "Impl" and the corresponding implementation class ended with "ImplImpl").
For some project, proj, the structure would generally look as follows:
proj- top-level, "pom" moduleproj-data- child "jar" module; the POJOs were named "{Something}DTO" and the interfaces "{Something}DAO"proj-services- child "jar" module dependent uponproj-data; the POJOs were named "{Something}VO" and the interfaces "{Something}Service"proj-web- child "war" module dependent uponproj-services; the POJOs were named either "{Something}Request" or "{Something}Response" and the interfaces "{Something}Controller"
These applications all suffered a similar trio of design smells: code redundancy (a high degree of similarity among the interfaces' methods and POJOs' properties within each module), low cohesion within each module (changelogs revealed that most non-trivial changes required changing code in all three modules), and poor orthogonality in the interface definitions (instead of a focused set of reusable general-purpose methods, there were many specialized methods with limited usefulness, a like getAccounts(AccountType t) method versus methods like getCheckingAccounts(), getSavingAccounts(), getInvestmentAccounts(), etc.)
I would describe this style of application architecture in this way: a hierarchicy of formal units, each having an appearance of usefulness but in reality providing little value and inhibiting overall productivity more than benefiting it.
This also happens to perfectly describe, in my opinion, the "architecture" of a bureaucratic organization, which happened to be the kind of organization that these applications' development teams belonged to.
If you might be thinking "Conway's Law" to yourself, I would have to agree. Conway stated it as such:
"Any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure."
In regards to addressing my confusion over how this anti-pattern could exist, I'd phrase it "Organization architecture begets application architecture", or even more succinctly: It's the organization, stupid.
On the topic of architecture, the term "architecture" is loosely-defined within the IT industry (and by "loosely" I mean "poorly"), but most would agree that it has to do with some foundational arrangement of inter-connected things. For application architects the "things" could be logical design layers; for system architects they could be VMs and database servers; for security architects they could be access points and firewalls. Generally speaking, the "things" which comprise the architecture are whatever happens to be important. Vyr
In 2003, Martin Folwer published an article entitled "Who Needs an Architect?" in which he described architecture as the things that are "hard to change" and suggested architects should seek to "remove architecture by finding ways to eliminate irreversibility in software designs.
Constraints are another view of things which are important. Some constraints are certainly things which are hard to change; others are self-imposed.
In the decade and a half since the article's publication, the growth of cloud computing and technologies like Docker have truly made it easier to remove the tehnical barriers inhibiting reversibility. Furthermore, practitioners of DevOps have been given numerous talks and written books about implementing Continuous Delivery practices. Combined, these have enabled software to be more easily changed.
There are two reasons for things being hard to change. The first is scale: the bigger the environment and the more usage it has, the more tricky things will be to change on the fly. The second
Anther example is a microservices architecture resulting from mostly-independent DevOps teams working within a shared organization providing automated CI/CD tooling, software-defined infrastructure, self-empowering corporate policies, etc.
Let's start with some various answers to this question already out in the wild.
- "Architecture is the base, irreducible, and immutable set of rules defining how a system operates." - http://wiki.c2.com/?WhatIsArchitectureAnyway
- Nothing is really immutable except for perhaps the laws of physics and materials science. Beyond that,
- "The software architecture of a program or computing system is a depiction of the system that aids in the understanding of how the system will behave." http://www.sei.cmu.edu/architecture/
- This is saying the architecture is just a model. If I want to know how a system will behave, I typically need to run it or exercise it with unit tests or something. Maybe a model could help me understand if there could be issues with scalability or HA, perhaps, but that knowledge comes from more than the model--it's specifics about the implementation technologies as well.
- "Architecture is the fundamental organization of a system embodied in its components, their relationships to each other, and to the environment, and the principles guiding its design and evolution. [IEEE 1471]"
- Great
- "architecture conveys a notion of the core elements of the system, the pieces that are difficult to change." -https://www.martinfowler.com/articles/designDead.html
- Garlan and Perry, 1995: "The structure of the components of a program/system, their interrelationships, and principles and guidelines governing their design and evolution over time."
Distilling this down, we have architecture as:
- a model of understanding
- an arrangement of inter-related things
- an immutable set of operational rules
- pieces of a system that are difficult to change
DevOps is another one of those concepts in software development that eludes concrete definition. IT leaders just know they want it and jump straight to the "How do you implement DevOps?" question.
A couple of ways to answer are:
- DevOps isn't something you implement, it's more of a cultural mindset that you initiate by tackling three aspects: people, process, and tools--in that order of importance. People are the most important, eh? It's the organization, stupid.
- If you mean you want to implement Continuous Delivery, then there are lots of different ways to go about it; the important thing is that you embrace change. "Yes, that's what we want to do." Are you sure you want to embrace constant change? If you do, why is there a Change Control Board that meets bi-weekly to approve all changes? Why do you have a bulky change tracking system independent of version control? Why do you
Continuous Delivery and have symbiotic relationships between development and operations sides of the house? "Yes, that's what we want."
Socialtechncail systems have sociotechnical architectures. The term "microservices" architecture has just as much to do with creating small-sized services as it does having an organizational structure of largely autonomous and development teams. Look
https://dzone.com/articles/scalability-introduction-for-software-engineers
When we are talking about scalability, we should see in front of our eyes people who are creating technology. An organizational structure, as far-fetched as it can be seen in developers' eyes, is actually rather important in that manner. The structure questions, such as what the team size should be, when considered and answered well, would cause you to be able to create better architecture. Now, when you choose your organization structure, you need to take into account a few considerations, and some of them are how much communication you need between people. You want the least amount of communication which will allow people to do a great job.
Obviously, if people need to communicate too much one with another, then not only will your business not grow fast and not be scalable, but also the architecture they create will be less optimally aligned with the solution you want.
http://www.hbs.edu/faculty/Publication%20Files/08-039_1861e507-1dc1-4602-85b8-90d71559d85b.pdf https://intellyx.com/2015/06/22/devops-insights-into-conways-law/ https://www.martinfowler.com/articles/designDead.html