Effectively avoiding pitfalls when scaling an engineering team


Published on November 06, 2020

complicated crossing

My favorite type of development is greenfield. Everything is brand new and you get a blank slate to work with. You get to choose the languages at use, which persistence technologies you will choose, which frameworks you will utilize. You define the architecture, select the appropriate hosting or cloud provider, and implement the initial code exactly how you want. It can be wrong or off but it is so early refactoring here is typically not a bad idea.

My favorite type of project keeps things small. A development team of five or less and the right supporting staff to help run the project. Every new team needs time to cycle through the various phases until your working smoothly but with a small team I believe it happens much faster. You can keep processes lean and overhead small. Things seem to just happen and you can iterate quickly.

This type of team or project could happen anywhere. Whether it be a couple of friends, a new project at work, or someone's startup. Because the size is small information flows freely, a consensus is easy to reach, and blockers can quickly be dealt with. Delegation and individual responsibility are also key here because the development team is small. Every team member gets to own a significant chunk of the overall work. Work gets completed quickly and the project rolls on as an effective and efficient team.

Teams and or projects can potentially stay in this mode forever which is great. However, when they don't and they opt towards growth you will encounter crossroads.

Being an engineering lead or architect you are well versed in the technical aspects of growth or scale. Tradeoffs must be discussed and appropriate changes took (at the appropriate time) to handle scale and load. Ideally, you plan for a stack that can handle your initial ideas but forgo premature optimization until you reach that need. These decisions and tradeoffs can be difficult because the information is not always clear and there is no right answer. Some proposals could be little changes for big wins while others look daunting.

On the other hand, perhaps the initial architecture choices are sound but need some gentle shifts to enable new features efficiently. It's not always a lot of traffic that is the technical challenge. However, with any technical challenges experience definitely helps guide your intuition. Your focus and perspective are on engineering and technology and you analyze and adapt to problems.

Maybe to get these new features out faster than your existing team could you raise money or you somehow augment your team to theoretically do more faster. This is a contentious point but I am avoiding it for now. No one project is the same but in my vast experience I have always had a team grow in waves rather than a big bang and 15 developers show up the next day.

Sooner or later when growth occurs you start arriving at crossroads in the non-technical areas of the project. These may or may not be in your control but awareness is key here when you do not have the responsibility yourself to make these changes. With awareness and some way to influence or make these decisions, you can then look for solutions and propose them to help the team through these turning points.

The funny thing is with these blindspots is that it's all a metaphor and they don't always stick out to you like an error in your logs or your application breaking when you submit a form. For sure if your team is now super dysfunctional you will feel it faster but if you are slightly going off the tracks or slowly picking up bad habits it's harder to see this.

Your perspective, awareness, and the way you work need to scale along with the team. If you attempt to keep doing what you are currently doing sooner or later you are going to fall into these traps or crash into what you cannot see.

It is impossible to be very specific and say watch for this or that because every team and project is different. However, what I have noticed is that as your team grows you reach a point where you will need to change to support the larger size.

  • What was implied must become concrete
  • What was general assumptions must be documented
  • Informal processes will benefit from being more formal and documented for others to follow.
  • Where perhaps you had loose and anything goes ways of working needs to shift to being more structured and documented so they can be repeatable
  • More people in a meeting at some point is ineffective and you will need to start focusing the audience
  • As a leader you need to delegate and dictate appropriately
  • Favor consistency and repeatable processes
  • Not all decisions can be or should be entire team decisions
  • Extra effort needs to be made on communication so everyone is on the same page

One concrete example I can think of:

Never have I ever been on a team that scaled that didn't encounter issues with how to approve pull requests. When you are small it is weird to lock a pr to x number of reviews. I don't advocate cowboy coding or pushing straight to the main branch but once you have greater than one developer you all collaborate and review pull requests. Things magically get done and work in a small team. Eventually, you get bigger and perhaps you say fine 1 person has to review and the process becomes a little more official. This usually works well for some time but then you get bigger and concerned people might not get patterns and maybe someday it becomes two. The other odd thing is figuring out who needs to look at these. Is it something you post in a chat channel and ask for? Do you use your tool to assign reviewers? Do you maintain a list of who is an appropriate reviewer ? by tool or technology? How do you keep that updated? What if assigning reviewers doesn't automatically inform people they need to do something? How long is too long to wait for inaction on a review?

All these things have popped up for me and the challenge here is there are a lot of ways to do this but you need to pick one and own it. Change if it doesn't work. Don't let individual teams do things differently. You want and need consistency. It is crazy to me how much time you can waste here when you all are engineers and know a general way pull requests get approved and merged but ask enough people and everyone feels differently.

As fun as the early, anything goes, we all own a lot phase is eventually it only works well for teams of a certain size. After you grow beyond this it's important to be aware of it and start shifting to a different style. Otherwise, your team suffers from an inefficient operating structure and you start negatively impacting your team members and their happiness.

If you liked this article, check out:

What do you think? Let me know on Twitter and follow me there for more development, programming, and general software engineering content.


© 2021 bkern, All Rights Reserved.

Built with Gatsby