The Impact Paradox

Reading time:

I’ve been thinking a lot lately about my career. Maybe this is my version of a mid-life crisis, but I’ve been thinking a lot about the impact I’ve had, and my prospects for actually making things better from here.

In the past five years, I’ve started the transition out of an individual contributor role, and into a varying mixture of product ownership, project management, and architecture (depending on the year). I was interested to move “up” so I could start grappling with larger challenges that I’d long seen but couldn’t do anything about. The obvious price - that I no longer have much time to write code - has had huge effects on my job satisfaction, with which I’m still coming to terms. Overall, it seems like there’s a paradox of impact here.

That is:

The more you can rightly say, “I did that” the less likely anyone will know what you’re talking about.

The more concrete your contribution, the less relevant it seems in the wider world. The more relevant the impact of your work on others, the less it’s a product of just your own hand. The degree to which big changes hinge on your own participation becomes smaller and more vaguely defined.

Here’s a good example. Recently, my team has been refactoring Indy - this huge, sprawling codebase that manages Maven and NPM repositories - from a monolith into a series of microservices. First order of business was to create an API gateway that would allow us to divert different functions to the new microservices as those come online, without presenting any API changes to the user.

The first version of this gateway had a huge problem, though. It used Quarkus with Resteasy and Vert.x to present a REST API to users, and Mutiny (also with Vert.x) to make requests to the backend services. This worked well enough, as long as it didn’t matter that we were buffering request bodies in memory. However, if you tried to download a large tarball from Indy, you could OOMKill the service.

To address this, I ended up rewriting the upstream request mechanism to use the OkHttp web client instead of Mutiny+Vert.x. When I finished the replacement, the whole service was incredibly fast and barely sipped at the JVM heap. I don’t get a lot of chance to write code these days, and ended up actually doing this work in my spare time, because it was a problem that just kept nagging at the back of my mind. I’d seen StackOverflow posts that hinted this simply wasn’t possible with Quarkus, but this really only made me more determined to find a way. Overall Quarkus is a really nice system; I absolutely love it!

I just want to pause here and say that after working in technology for 20+ years, there is still almost nothing in my professional life that compares to the absolute high of solving a problem with a really nice piece of code.

So, if I like this so much, why did I move away from coding? Well, as proud as I am of that code, and as well as it works, it only solves a small problem. Most people I work with probabaly didn’t even know the problem existed, and it does nothing to improve the clarity of our path toward more scalable technologies. When you’re coding, you’re rarely thinking about how that new feature will position your team for future work. In fact, it’s usually a mistake to try to think that broadly. The result is often software that’s so flexible it can hardly be used by mere mortals. This kind of thinking is how new scripting languages come into the world.

I really want to address problems that will inhibit our agility in the coming year or two. I want to think about how the shape of our codebase affects team health, and how we can use that relationship to improve the health of our teams. I like trying to see around the corners of technology, and making plans to be ready before we run into a problem. But this work is very different from writing code, and those differences aren’t always obvious.

As an architect, I rarely own a new achievement in the sense that an engineer does. Rather, any success in my goals necessarily also involves my team (who probably did all the coding). At the end of the day, I don’t have something running in CI that is an obvious improvement. My days are filled with conversations that often have no obvious conclusion, only “enough to get started”. And writing things that I’m not sure anyone ever reads. As an architect, progress is much less obvious or concrete. But if we do achieve a major milestone, non-technical people understand it. So, claims I make when talking to non-technical people feel vaguely like embellishment.

I salve this discomfort by looking back at the progress we’ve made. In growing a team and then growing it into a couple of teams. In building a cluster-enabled version of our monolith, and then refactoring half of that into microservices. We’ve come a long way, really, since it was just me writing code for Indy.

I think that’s the real lesson here, at least for me. In order to grow beyond individual contributions, I need to think in terms of things that we were able to achieve together as a team; places where maybe I provided a specific input to the process, but I didn’t do it all. And that makes sense, because these are big things we’re talking about. My accomplishments now are inseparable from the team I work with; they really can’t be talked about as separate work, because they’re just ideas that someone else brings into reality. Without my team, they’re just dreams.

I don’t want to go back to being just an individual contributor, at least not for now. But I’m definitely not yet comfortable in my skin as an architect or technical lead. Maybe that point where you become comfortable in your role is actually the sign that it’s time to change again.

I’ve been thinking a lot lately about my career. Maybe this is my version of a mid-life crisis, but I’ve been thinking a lot about the impact I’ve had, and my prospects for actually making things better from here.

In the past five years, I’ve started the transition out of an individual contributor role, and into a varying mixture of product ownership, project management, and architecture (depending on the year). I was interested to move “up” so I could start grappling with larger challenges that I’d long seen but couldn’t do anything about. The obvious price - that I no longer have much time to write code - has had huge effects on my job satisfaction, with which I’m still coming to terms. Overall, it seems like there’s a paradox of impact here.

That is:

The more you can rightly say, “I did that” the less likely anyone will know what you’re talking about.

The more concrete your contribution, the less relevant it seems in the wider world. The more relevant the impact of your work on others, the less it’s a product of just your own hand. The degree to which big changes hinge on your own participation becomes smaller and more vaguely defined.

Here’s a good example. Recently, my team has been refactoring Indy - this huge, sprawling codebase that manages Maven and NPM repositories - from a monolith into a series of microservices. First order of business was to create an API gateway that would allow us to divert different functions to the new microservices as those come online, without presenting any API changes to the user.

The first version of this gateway had a huge problem, though. It used Quarkus with Resteasy and Vert.x to present a REST API to users, and Mutiny (also with Vert.x) to make requests to the backend services. This worked well enough, as long as it didn’t matter that we were buffering request bodies in memory. However, if you tried to download a large tarball from Indy, you could OOMKill the service.

To address this, I ended up rewriting the upstream request mechanism to use the OkHttp web client instead of Mutiny+Vert.x. When I finished the replacement, the whole service was incredibly fast and barely sipped at the JVM heap. I don’t get a lot of chance to write code these days, and ended up actually doing this work in my spare time, because it was a problem that just kept nagging at the back of my mind. I’d seen StackOverflow posts that hinted this simply wasn’t possible with Quarkus, but this really only made me more determined to find a way. Overall Quarkus is a really nice system; I absolutely love it!

I just want to pause here and say that after working in technology for 20+ years, there is still almost nothing in my professional life that compares to the absolute high of solving a problem with a really nice piece of code.

So, if I like this so much, why did I move away from coding? Well, as proud as I am of that code, and as well as it works, it only solves a small problem. Most people I work with probabaly didn’t even know the problem existed, and it does nothing to improve the clarity of our path toward more scalable technologies. When you’re coding, you’re rarely thinking about how that new feature will position your team for future work. In fact, it’s usually a mistake to try to think that broadly. The result is often software that’s so flexible it can hardly be used by mere mortals. This kind of thinking is how new scripting languages come into the world.

I really want to address problems that will inhibit our agility in the coming year or two. I want to think about how the shape of our codebase affects team health, and how we can use that relationship to improve the health of our teams. I like trying to see around the corners of technology, and making plans to be ready before we run into a problem. But this work is very different from writing code, and those differences aren’t always obvious.

As an architect, I rarely own a new achievement in the sense that an engineer does. Rather, any success in my goals necessarily also involves my team (who probably did all the coding). At the end of the day, I don’t have something running in CI that is an obvious improvement. My days are filled with conversations that often have no obvious conclusion, only “enough to get started”. And writing things that I’m not sure anyone ever reads. As an architect, progress is much less obvious or concrete. But if we do achieve a major milestone, non-technical people understand it. So, claims I make when talking to non-technical people feel vaguely like embellishment.

I salve this discomfort by looking back at the progress we’ve made. In growing a team and then growing it into a couple of teams. In building a cluster-enabled version of our monolith, and then refactoring half of that into microservices. We’ve come a long way, really, since it was just me writing code for Indy.

I think that’s the real lesson here, at least for me. In order to grow beyond individual contributions, I need to think in terms of things that we were able to achieve together as a team; places where maybe I provided a specific input to the process, but I didn’t do it all. And that makes sense, because these are big things we’re talking about. My accomplishments now are inseparable from the team I work with; they really can’t be talked about as separate work, because they’re just ideas that someone else brings into reality. Without my team, they’re just dreams.

I don’t want to go back to being just an individual contributor, at least not for now. But I’m definitely not yet comfortable in my skin as an architect or technical lead. Maybe that point where you become comfortable in your role is actually the sign that it’s time to change again.

Written on November 13, 2021