Does more information always mean a better outcome? It all depends on the context.
Take baking. When you make a cake, your ingredients go through a bunch of chemical changes and processes. If anything goes awry, the cake won’t come out right. But that doesn’t mean you need to understand what’s happening at a molecular level. You just need the high-level knowledge to spot where you made a mistake and fix it for next time.
Unless, of course, you operate a cake factory. It’s still baking — the same reactions are taking place — but at this scale you would need reliable ways to monitor chemical processes and variables. Without granular information, you won’t be able to find the root of problems fast, hone your production processes, and avoid expensive mistakes.
The same is true of build observability in any dev team.
Say you’re a small organization or team, and you just have a few relatively simple builds to worry about. If a build fails, it probably won’t be too hard to retrace your footsteps and figure out where you went wrong. It might be a simple case of, for example, scrolling through manual logs, asking your colleagues which one of them changed a piece of code, and correcting it.
That’s not to say you don’t need any build observability. You still need to know where the problem is, after all — especially if you ever do want to scale up. But in this case, rushing into adopting some of the more complicated tools out there would be the equivalent of installing state-of-the-art, NASA-grade temperature and humidity monitoring in your kitchen because you’re worried your chocolate ganache might melt. Simpler tools will likely give you all insights you really need, without confusing you with ones you don’t.
More cooks, more problems
For larger teams though, this basic level of observability probably won’t give enough nuance and depth to maintain growth. The more complex your projects, the larger your team, and the more builds you run, the harder it will be to spot a mistake buried deep in your documentation, especially if you’re trawling text logs instead of using tools that help visually represent your project.
Rather than responding reactively to problems and figuring out how to avoid the same mistakes next time, at this stage in your growth you’ll need to be strategic about collecting granular, day-to-day insights and using them to address bigger problems with your build.
Are you seeing the same bugs and bottlenecks over and over? Are you under-utilizing your processing power? Or running too close to maximum capacity, and risking a crash? Better to spot the signs of strain and tackle the problem before your overloaded systems crack like a crème brûlée.
Adjusting to everyone’s palate
What happens if your team is so big that your dev and DevOps colleagues don’t even work in the same department — and need totally different kinds of information to do their jobs effectively?
Imagine that cake factory again: the master baker overseeing production in one part, while your technical or engineering team are set up in another. Everyone wants to make sure the end consumers get the best possible product, but much like your dev team and DevOps teams, they look at the challenge differently.
Much like developers, the bakers are mostly worried about checking the final product to make sure it’s right — and if it’s not, establishing the high-level reasons why (e.g., are the ingredients okay? Has a machine broken down? Is someone on the production line messing up their task?).
The engineers and technical leads, on the other hand, are more like your DevOps team. They’re worried about the finer details of how the machinery is working, or what other pesky conditions might be setting off unplanned chemical reactions that interfere with the output.
They’re looking beyond the “why” — e.g., “this component keeps crashing” — to the more pervasive underlying conditions and the intricacies of how each element works. That’s because they need to know exactly what they can do to ensure it doesn’t fail, and how the overall operation can be continually tweaked to maximize performance and avoid further issues down the line.
For the perfect strategy, season to taste
When it comes to making a build observability strategy for your dev and DevOps teams, this means you need a way to capture both types of insights: the headline, macro-level insights for your dev team, and a consistent flow of detailed, precise information on exactly what’s going on under the hood for DevOps. That way, they can complement each other perfectly, working together to catch problems and speed up builds, ensuring they have the CPU capacity they need to avoid unplanned downtime, and making sure end users get the best possible experience as a result.
In a nutshell, cooking up the perfect build observability strategy for your team depends entirely on your organization. It will be shaped by factors like the size of the company or team, the way it’s structured, and how big and complex your builds are.
That said, it’s always good to give yourself room for growth. Even if you’re a small outfit with simple codebases that doesn’t need (and doesn’t have the time) to monitor every tiny scrap of data, you still need easy, reliable ways to figure out where things are going wrong in your builds. Otherwise, how will you scale?
This is where strategy comes in.
Whisking up a long-term winner
Whatever the size of your outfit, you shouldn’t just be thinking about scraping by now. You should also be considering how you’ll future-proof your processes and systems.
If you have the level of build observability you need, it should be easy for you to identify past errors and more persistent problems, such as issues with compile times, utilization rates, system capacity, and processing power on your network, so you don’t hit a wall on your next, more ambitious build.
And if you’re already a giant operation, the stakes are even higher. Without these insights, it will be hard to stay nimble and competitive, especially if up-and-coming contenders already have a stellar build observability strategy in place. Better to figure out how to handle the heat before you get nudged out of the kitchen.
Enjoyed this post? There are more insights where this came from! For detailed advice on developing your build observability strategy and picking the right tools to help you, download the full whitepaper here.
