When a startup is just getting off the ground, it's not uncommon to build prototypes and put them into production. But sooner or later, you have to pay the price.
I have a client which launched a business with devs working in silos. Some of these silo projects are still more effective as silo projects, but others are getting to the stage where they feel the predictable pains that are associated with this style of development.
So my client, as a company, needs to dismantle the silos they've built — but without transitioning away from the model entirely. It's too early for that in some parts of the company, but overdue in others. So they need to make the transition partially, and they also need to grapple with the difficulties inherent to changing a dev culture in any way.
Three Problems With Silos
One of the major problems with a silo approach to development is the "not my fault" factor. When systems are siloed, they become synonyms for the people who work on them. It's very difficult to create a blame-free culture when identifying a flaw in a system is almost the same thing as identifying a mistake that could only have been made by one particular individual.
Another problem is workload imbalance. Say two engineers start on the same day at a company which uses silos. The dev manager assigns Engineer A to run a clearly documented, well-maintained system that is hardly ever even used in the first place. Engineer B gets responsibility for an error-prone system which sees a huge amount of traffic every day, and mishandles most of it. So Engineer A is always relaxed and frequently even bored, while Engineer B tears their hair out in panic on a regular basis. Forget about a blame-free culture; it's hard to have any kind of consistent dev culture if developers never have the same workday in terms of hours, stress level, or urgency.
These are culture problems. Silos also create technology problems. The big one: business logic duplication. When multiple siloed systems each implement some aspect of the business logic, your best-case scenario is that there is no one source of truth. Your worst-case scenario is that the implementations will contradict each other.
How Do You Fix It?
Before going any further, I should clarify that I'm not necessarily describing exactly the problems my client has. These are just problems inherent to silos.
Changing the code base is actually the easy part here. If you have three systems which each implement the same business logic, you choose one and make it the source of truth. Then you have the other two get their information from the canonical source.
But how do you change a dev culture? Say you're a CTO and you decide all your engineers are now going to learn about all of your systems. Hooray! Silos will end!
But one side effect of the workload imbalance problem is that if you're an engineer and all your work happens in an easy silo, and you see other engineers stressing out about their difficult silos, you might want to stay put in that easy silo without learning anything about the difficult one.
Meanwhile, from a leadership perspective, if you've invested time and energy developing one particular engineer to the point where they're an expert in one of your systems — call it System X for the sake of argument — it might be pretty expensive to train them on some other system, or to train some other engineer to the point where they're an expert in System X also. Especially at an early-stage company, you might not have infinite time for your team to do that.
Going back to the individual contributor's perspective, every moment your company has any kind of "not my fault" factor in place, it reinforces silos, because it creates a disincentive to learn new things. If systems are synonymous with the people who run them, admitting you know anything is volunteering for blame.
Disincentives to learning are poison for technology companies. How do you counteract this poison? What's the antidote?
The Solution: A Game Show
Patrick McKenzie recently wrote that:
In a way, every scaling startup is an experiment in empirical microeconomics research on “What parts of the typical corporate form are necessary and which are pageantry which we only keep around due to anchoring, the sunk cost fallacy, and tradition?” Every time a startup bites the bullet and hires a VP of Sales, a lifecycle email copywriter, a retirements benefits administrator, or a cook, count that as a published result saying “Yep, we found this to be necessary.”
Consider this blog post an argument that every startup needs a game show host.
Silo-ed development has self-reinforcing aspects, and humor is a good way to break those types of spirals. So I created a meeting which functioned like Whose Line Is It Anyway, the improv TV show "where the points are made up and the scores don't matter." The format featured three games:
- What Do They Do?
- How Does It Work?
- Pretend You Know
The first two games were quiz games. I asked a bunch of questions, everybody came up with answers, and at the end we compared everybody's scores. For What Do They Do?, I put up slides asking questions about what individual developers did in an average workday, and for How Does It Work?, I asked questions about the various systems at the company. Right answers were worth a point each, and you'd get a half point for answers which were not totally right but not totally wrong either. In keeping with the goal of a blame-free culture, there were no penalties for wrong answers.
I named this meeting The Wrong About Everything Party. The name was actually a mistake; I had intended to present a demo documenting all the systems at the company, and fully expected every other engineer in the meeting to correct me about some detail in the systems they knew. I felt it was more important to have a description which was full of errors than no description at all. By the time the meeting rolled around, though, I had changed my focus from telling people what I knew to asking them what they knew.
As an aside, I learned this at a previous company. I and a few other engineers had built a system using new React features that most other developers weren't familiar with yet. I did a "Lunch & Learn" where I gave a presentation about this system, and everybody enjoyed it, but nobody actually learned how to work with the system. This was another silo risk.
That time, I mitigated it by doing a followup "Lunch & Learn" which was really just a mob programming session. Instead of giving a presentation, I posed a problem: we need to modify this aspect of the UI. What file do we change first? Getting people to take action was way more useful, in terms of knowledge sharing, than just telling them stuff while they ate potato chips.
Anyway, The Wrong About Everything Party went really well. There were a couple introverted engineers who worked with more systems or languages than the rest of the team realized. This made it easy to surprise the rest of the team with a few gotcha questions, and it seemed gratifying for those engineers to have their expertise recognized. Obviously, directing recognition to introverts is a balancing act, but the light-hearted, silly mood of the meeting made the balance easier to handle. It also made the rest of the team ask themselves questions like, "Am I underestimating this person?" It's always better to have somebody ask themselves that question than to just straight out tell them.
There were also a bunch of technical problems which people had been wanting to fix for a while, and sharing this burden with the rest of the team, even if only verbally, made it less stressful. At least, that's how it looked to me, although of course I'm biased. More concretely, people were laughing and volunteering information about how various systems worked. After the meeting, people said they wanted another one, and came to me with other silly factoids that could work as questions next time.
Why It Worked
Gamification, as a trend, got way out of hand, but game design is a good way to look at structuring organizations, because in either case, it's all about setting up incentives and encouraging interactions. That's why there were no penalties for wrong answers, and you could get half a point for an answer that was only half-right. It encouraged guessing.
Any guess in this context was a statement, according to the rules, but since you would make your guess and then pay attention to whether you guessed correctly or not, every guess also functioned as a question. When the problem is isolated knowledge, it's good to incentivize asking questions.
And as far as a blame-free culture goes, this game put us all in a situation where it was possible for a developer to share knowledge about a system without claiming that system as their own, and making it a synonym for their name.
I had a separate game called What Do They Do? for three reasons. As stated above, silos have a problem where your silo overlaps with your identity too much. But if there are two separate games, one about people and the other about systems, that pushes you to differentiate between people and systems. Most importantly, getting people to experience genuine curiosity about each other is a nice place to start if team-building is one of your goals.
Pretend You Know was a filler game that I threw in at the last minute, figuring I should have some backup just in case we got through all of the quiz-style questions in the first two games. In practice, we didn't need it, but we played a round of it anyway, and the meeting went a few minutes over as a result.
The rules of Pretend You Know are simple: two contestants, one referee. Each contestant takes turns describing to the referee how a system works. It can't be a system that either contestant works on every day; it has to be a system that the referee works on every day. It again requires you to do a lot of guessing as you go, and although it's obviously very silly, it's fun to watch.
In general, The Wrong About Everything Party went really well, and I ran another one a few months later. This isn't the kind of meeting you necessarily want to have every week; it requires some prep and it's not going to feel like a party if it happens all the time. We don't have Halloween every Thursday.
Silliness Can Be Useful
There's plenty of research that shows people learn more when they're having fun. Moreover, a serious conversation about an unfamiliar code base is inherently difficult. How do you know what parts of the code to talk about, especially when you work in one silo and the other people in the conversation work in other silos? By definition, you don't know what the other people in the conversation need to learn. You know your code has some responsibilities and business rules in common with theirs, but you don't know how many or how much.
Prioritizing topics in that context is difficult. It's also hard in that context to find a good balance between a high-level overview and a deep dive. Last but not least, the way that silos generate blame means that any serious conversation about technical debt runs a high risk of finger-pointing, which is never productive.
My solution was to prioritize silliness. That is to say, if I had three things I could talk about for any given system, but I had to choose one, I would choose the silliest one.
Sometimes it's impossible to have a serious conversation about technical debt and have a good time. That does not mean that your only option is to have a serious conversation about technical debt and have a bad time! You could also just have a silly conversation about technical debt and have a good time. In practice, all that really matters is that the developers share information and gain an improved understanding of the systems they run.
People often praise the courage to make a tough decision, but it can be a false dichotomy. People learn more when they're having fun, and it'd be wildly counter-productive to incentivize finger-pointing when your ultimate goal is to upgrade how a team collaborates. Most importantly, if you get a bunch of good engineers to agree that an implementation or a design is silly, you've basically guaranteed that they will fix it.
That's begun to happen, although of course The Wrong About Everything Party is not the only reason.
