A wise man once said "If you are going to dive head-first into DevOps, better pack a parachute". Aside from the fact that this man was me, the point is still valid.
Infomentum had started maturing into a DevOps culture, which involved the developers using tools normally associated with DevOps to solve development problems. We were calling it DevOps for several years, even though the actual DevOps culture, practices and support were not in place. The changes made were effective at enhancing the development lifecycle, but were completely separate and sliced off from the rest of the organisation. This meant that infrastructure needed to catch up so that the DevOps practices could be fully utilised throughout the company. The major issue with DevOps is that it really is all or nothing; as soon as you have segregated processes and half-utilised technologies, the power of DevOps is completely negated.
For example, if you used a configuration management (CM) tool to automate the installation of a few base packages, you'd need to be on every server in an environment - only to then have to manually install other packages, change the version of existing packages, and remove others. This defeats the point of using a CM tool in the first place. The servers you build using CM should always be in a known state which is stipulated in CM. For example, if I were to build a server using CM and someone deleted it, I'd be able to rebuild it using the existing configuration to let it return back to the state it was in before getting deleted. This means that, in an ideal world, even if you're going to manage the packages installed on a server using only CM, you need to manage all the packages and ensure no manual changes are made to them.
These were exactly the type of issues Infomentum was facing with the "half-in half-out" approach to DevOps. If an organisation decides to take a DevOps approach then a plan needs to be put in place. With Infomentum, it was all about discovering what problems the organisation was facing and how/if they could be resolved by adopting this approach. We then needed to create a road-map of how we'd implement the new changes; and most importantly, how we'd shift our employees' perspective, to ensure the whole organisation agreed to migrate to a DevOps approach. This is where the opening quote comes into play. You can't just throw tools at organisational and cultural problems, and expect to create a DevOps culture. This may sound obvious, but seems to be a frequent issue that organisations face when they try to implement DevOps.
Now, I guess your big question is "Well, what did you do to mitigate and overcome the issues you were facing?". We started with the fundamentals; the first step was to ensure that we had the correct foundations. After we'd identified the issues we were facing, both as an organisation and from a technological standpoint, we had to plan what needed to change and create the requirements for what needed to be done. This involved getting members from different teams, such as the development and infrastructure team, into a meeting room, to come up with vision statements, create action plans, and break them down into epics. We came up with the following:
- Logging and monitoring
- Configuration management
- CI/CD
- Continuous testing
- Secrets and password management
- Version control
- Client management
- Security
- Backup and recovery
- Training and documentation
- Migrating current infrastructure
We decided that these epics needed to be in place to visualise what the first version of our DevOps framework would look like. The next step was to work out an appropriate order, and prioritise which of these framework V.1 end-goals to begin working towards.
We decided that a good place to start was logging and monitoring, and we chose to build this using configuration management. This created an obvious overlap between epics, but that was inevitable with such high-level goals. The important part of this is that we set the standard going forward, ensuring that everything we build is replicable, secure and manageable. Our configuration management is version-controlled, idempotent and end-to-end.
By splitting this epic into tasks and sub-tasks, we were able to plan appropriate sprints which we could allocate the work to, to begin moving towards our first goal for the DevOps framework.
This was the start of Infomentum's DevOps journey. I'll continue to share more about it in future blog posts.
