Kent Beck recently stated that Agile had failed. Agile itself is not a failure, but the push, by some of the best developers and software architects in the world, for the industry to adopt Agile, had failed. The failure was likely due to there not being enough tech people pushing for "Real Agile". All too often organizations adopt a fake version of Agile, and not the real deal. In that sense it has totally failed. I would like to contribute to trying to bring back the push for Agile, as it is really critical for organizations, but also the world. The pace of technology change is affected mostly by the life cycle mindsets that folks in the tech industry adopt. Since technology betters everyone's lives, I would like to contribute to that, at least in a small way.
For the past 18 years I have been an advocate of Agile mindsets, even before I had heard of Agile. This notion of Big Bang deployments, no automation in testing or deployments, no real collaboration, etc., has always been foreign to me. From day one of my career I advocated for unit testing, paired programming, automatic deployments, code reviews and knowledge sharing, etc. To me it just seemed essential, and I've always been quite surprised when others do not agree. We want to deploy a system with "uncertainty" because everyone likes to come in on the weekend to support it? Yeah, questions like that really highlight the thinking "Right, we want certainty".
Let's take a look at the Agile Manifesto, and how it applies to the modern IT industry. Maybe this is preaching to the choir, and just serves as affirmation for you. Maybe this is new to you, and you've been wondering what DevOps really is. In either case, I hope you find something useful here. This is intended to be a series of posts, starting with a basic outline tied to the Agile Manifesto. I hope this can serve as a small contribution to revitalize the trust in Agile.
This article series applies to both technical folks and IT managers, as they contain generic concise overviews, that most people in those groups should understand.
Let us walk through the first page of agilemanifesto.org as the guide. We'll move on from there, with further articles, and look into the Twelve Principles of Agile. When the series is complete, I will compile it into a larger single article, with links to the first article if someone wants to read it one at a time.
Individuals and interactions over processes and tools
Agile covers delivery of software, but it covers it from the point of people, and their interactions first. This is largely because it is people who determine what needs to be done and how to do it. If we are not including the required people in our processes, and ensuring they all receive the feedback they need to make the best choices, then their choices will be sub-par.
Agile is very specifically not a methodology. Methodology has nothing at all to do with Agile, as Agile is not prescriptive in any way. It never tells us "do this and you'll be agile". Scrum may seem like it tries to do that, albeit poorly, but it's not based on Agile at all. Scrum came before Agile, and is very much about strict processes, and not as much about people and interactions as Agile is. Agile only prescribes the mindsets of successful Software Development. An agile mindset approaches each bottleneck (usually a non-agile process causing it) looking for a place to inject *agility. That will be different for each organization, as each organization is unique. Each organization, and the teams within them, may come up with their own methodologies of how they approach problems, but Agile does not prescribe those methodologies. Agile is not something we can "be", agility is something we want to "have more of". To "be" implies an end state, and that's not what Agile is about, as it has no "end state".
Many times, people believe Agile is cowboy world, with no rules, etc. Agile is not against processes or guidelines, it advocates for processes that work well with people and create efficiencies in the work we do. I would argue that Agile is more about guidelines than rules. Rules have close to no agility, while guidelines help keep you focused and consistent. An Agile team helps keep individuals from going outside the guidelines unnecessarily, or without justification. Whenever there is reasonable justification it becomes obvious that going outside the guidelines is necessary, and that is agility.
DevOps has some essential concepts contained in it, which define things in a little more detail, about how an
organization should go about being more agile. But even with DevOps they are defined in general terms such as ``
store documentation with your software in the same repoorbuild your pipeline in the same repository as the
code`, or hundreds of other concepts. They are still just concepts, without the "how", but define more of the "what".
They are prescriptive concepts in that without them, we will not have as much success.
Working software over comprehensive documentation
Documentation gets outdated very quickly. If that is happening, one should consider whether it is documentation that should not have been written in the first place. In the modern age of DevOps, us Agile geeks would say that everything should be automated. To a large degree, the automation (infrastructure as code, unit tests, etc.) is itself the documentation. It will be kept up to date and does not require a human to even think about keeping it up to date, as the system literally fails if it is incorrect. If we write a deployment script correctly, we will have DevOps pipeline failures long before production, and our software will never get to the intended environment. This is good because we immediately know it is wrong.
Other than the automation, what do we document? Well, really anything that can give people a good overview of the system. That might be business overview documents, architecture documents, documents describing the basics of how to get the system running, and documents on Standard Operating Procedures for diagnostics. We should aim to minimize the amount of documentation, or it will get out of date; it will not matter what, but at least these types of documents add value, even if they are slightly out of date. We also want to focus on documentation that won't change a lot over time. If every time we are changing the system, we need to also change the documentation, then we shouldn't be documenting that thing. Instead, we should be automating that thing in some way.
Working software should be happening from day 1. Quite literally deploying an application to production, on day 1, or as soon as humanly possible, is the only way to go with modern Software Development. This is the DevOps way, and creates this feeling of success daily, keeping everyone motivated to keep producing, and it becomes very addictive.
Customer collaboration over contract negotiation
This one is pretty simple. The waterfall approach does not work, this has been proven time and time again. Putting time and energy into massive "independent" phases of requirements gathering, analysis & design, development, testing, and finally deployment, and in that order, causes uncertainty. How can we know if our system is going to even be what we need, or work when it hits production, before we build it, test it, and deploy it? We cannot know if we are not collaborating with customers and getting feedback throughout all phases of a project. And the best way of doing that is iteratively, in that all phases feedback into each other.
Responding to change over following a plan
In many respects, this one really ties into all the rest of the values on the main page of the Agile Manifesto. In essences, this is a summation of the previous values, in that if we are adhering to the previous values then the results will usually be that we are "Responding to change", and that is what we call "agility". If we approach software projects with a waterfall mindset, it results in long-drawn-out phases of requirements gathering, analysis, design, test, deployment, and a great deal of difficulty in conveying feedback between the phases. In Agile, when adopting tight feedback loops between all cycles we receive the needed feedback for each phase repetitively.
Doing things iteratively results in more certainty because each piece we build is small and easy to comprehend, design, test, deploy, maintain, and easy to know if it is what the business needs. It is important to understand that doing big design, development, and deployments, frequently results in a tonne of waste, simply because we end up writing and deploying features that are not used and/or get the feature completely wrong. Put all this aside for the moment, and think about the bottom line, going big fails! It fails in budgeting, it fails in time to market, and it fails people. It fails in almost everyway that we think of.
We will get into more details in follow-up articles. If you haven't read my post on Agile iteration, it might be good to do that now, as it can help with understanding the iterative cycles.
A Word from The Chief Engineer at Space X
I have always wondered how Space X has been so successful. We have all watched them succeed, reducing the cost of space exploration time and time again. What I did not realize, until I watched an interview with the Chief Engineer, is that they are ultra-Agile. Of course, Agile applies to software development only, but the principles are from the lean movement, and principles many times apply to all of life. In the interview, this Chief Engineer describes how they achieve these great things; seriously go watch 6 minutes and 30 seconds of that now, before continuing. As you watch, you will notice some key points to success…
If a design is taking too long, the design is wrong!One of the fundamental **errors** made in advanced development is to stick to a design even when it is very complicated, and to not strive to delete parts and processes!Everyone is the Chief Engineer; this is extremely important!Everyone must understand, broadly speaking, how all of the components work.In Agile this would be called a cross functional team.Sub-system optimization without considering the whole is backwards.Product errors reflect the organizational errors.The constraints you are given, from another component built by another team, are guaranteed to be some degree wrong.Question the constraints. It does not matter if someone building it won a Nobel Prize, even Einstein made mistakes.The biggest mistake smart engineers make, is that they optimize a thing that should not exist.Honestly, the thing I love most about this guy, is his humility. Humility fortifies collaboration! Not collaborating is torture to the ones who love success.