I was devastated.
Two days before the launch of Docker Security Scans, we found a bug. The new feature, which would show security vulnerabilities on Docker images, wasn’t showing the same number to users that we had on the backend. It quickly became clear that there was no simple fix– the issue was coming from a part of our system that the team didn’t understand. This part of the system was critical. If we made a change, we’d potentially break a number of other features. The timeline to gain an understanding and fix the bug? One month. Time we didn’t have. Docker was the fastest growing developer technology ever. But the company had struggled to develop a scalable revenue model. The business was counting on this new product release to regain lost trust with developers by showing that Docker images would not introduce vulnerabilities into their systems. Once that trust was regained, Docker would be back on top, allowing us all to keep our jobs.
But a successful product release was slipping out of grasp. The entire future of the company was in jeopardy.
Too much code and no one to understand it
This instance was not an anomaly. This is a rampant problem for developers. With constant turnover at tech companies, engineers are always having to acclimate to new systems.
We are also shipping more code than ever before. According to SourceGraph, we are shipping 10x as much code as we did 10 years ago. Developers are managing 100x the amount of lines. Because so much code is being shipped by so many people so quickly, developers have to continuously acclimate themselves.
As a result, engineers build things without a complete understanding of how the system works. Why? Because it simply takes too long. . It’s easier to build an entirely new system alongside the old system than to gain an intimate understanding of the legacy system and just fix the root cause problem. Worse, it’s even harder to feel 100% confident that you understood everything even if you did know how the system worked.
Always be understanding
Having difficulty understanding how part of a system works causes longer development times, unlaunched features, and a lot of frustration. We’ve accepted it as an “accepted truth” as an engineer, but surely there’s a better way.
Shortly after my experience at Docker, Josh Leven, my husband and co-founder, sent me a video of Brett Victor’s Inventing on Principle. The video hit me like a ton of bricks. This sentiment, in particular, struck me:
“…to a large extent, the people that we consider to be skilled software engineers are just those people that are really good at playing computer.
But if we're writing our code on a computer, why are we simulating what a computer would do in our head? Why doesn't the computer just do it... and show us?”
In his video, Victor mentioned a few points that were pointing to the root cause of the problems I was seeing. In particular, Victor concluded that we require developers to build a mental model in their head– one line of code at a time.
If we had a way of giving them visual tools– showing data, comparisons– then they could be put on the fast track to building these mental models, ultimately helping them to develop efficiently.
The seed of an idea: Visualizing codebases
Victor’s video gave me an idea. If we could automatically visualize entire codebases, we could understand code faster and easier, develop new features more confidently, make better plans, and understand context within architecture.
Josh and I agreed it would be difficult to pull off, but since the computer knows what it’s doing, it should be possible to visualize the codebase on screen. Josh dismissed the idea as unrealistic reasoning that if my idea were possible, someone would have done it already. My response? We’ve got to give it a try.
From there, the research began. We combed through academic research papers and looked into existing tools in an attempt to find out why this hadn’t been done before.
We found out that it had been attempted, but those creators had wanted to improve security or create dependency tracking. Our goal was to help developers understand their system before writing any code. After all, developers spend much of their time simply acclimating themselves to new codebases.
It was time to get to work.
Validating our idea for a product & becoming a company
Josh and I knew that creating a visualization tool for codebases could be a game changer, helping developers build mental models in record time.
We split duties. He built a few prototypes and I went out and talked to people to find out if this problem we identified was as big as we suspected.
I talked to 50+ people who all groaned when I mentioned this problem, proving this was worth solving. Every single engineer I spoke to had the same issues– it took anywhere from 3-12 months for a new developer to understand a new codebase. It was a huge time sink that made it difficult to get new features out the door.
Here are a few stories I heard:
- One person worked as a software architect, meaning their entire job was to understand how the system worked. After one year trying to understand his company’s codebase, he only gained a 20% understanding on one product.
- Josh was in charge of engineering at his company and had been the first engineer on the team. After seven years working with the system, it took him three full months to document how their system worked.
- One developer told me that he put someone on his team to try to map out the entire codebase but the effort was futile because the code changed so quickly that by the time he was done he’d have to go back and do it again.
What are we doing at CodeSee?
My frustrating experience at Docker was the beginning of creating CodeSee. Our vision for CodeSee was to create a tool that would allow developers to see how the system actually worked in the first place. The problem of “we don’t know how every part of our system works” would not exist.
If we’d had CodeSee at Docker, we would’ve known how the system worked. Not only would we have been able to fix the bug, but we would’ve had a better understanding of the code in the first place. We would’ve built a better feature from the start.
CodeSee’s mission is to help teams master the understanding of large codebases. We want to finally bring some clarity to code. We’re still in the early stages, but our products will change the way we build an understanding of technical systems. We’ll no longer need to spend months building mental models or guess how connections work between services.
With CodeSee as the solution, developers will spend less time getting acclimated and more time on the work they love– tackling interesting technical problems and making a positive impact on the world.