I have been advising many customer CTOs and VPs with their product strategies, product roadmaps and modernization efforts. In the past I have also led new product developments, and have had engineering ownership of a mature banking product with over 500 customers. Looking back at what I learnt from all of this, I realize that I see a clear trend here - the more success a product has had in the recent past, the higher the chances that engineering is nearing a dead-end. The higher the past success, the more difficult it is to achieve the next leap in engineering for future success of the product. And, on top of that, the longer you wait to take the next leap in technology and engineering, the worse it gets.
Why does this happen?
Let us assume you head engineering, and have started building a new product. You have a clear vision of what the product needs to do, and how you are going to achieve it. The design, architecture and roadmap of the product is based on this initial vision. The choice of technolgies and tools is similarly based on current needs and current availability. Everything goes well in development. You release the product into the market and sit back and relax, expecting to keep working on the roadmap at your pace and priority. Suddenly, your product picks up! You have new customers signing up every day and guess what, your plans are hijacked. The business wants to capitalize on the momentum and starts pressurizing you into providing new features faster, features that you have never planned for. Customers start getting pushy about their defects and their feature requests. The load on the system keeps increasing dramatically. You hire a larger team, go with the flow, start churning out releases by the dozen, add many new features, increase the infrastructure footprint, integrate with a bunch of partner products....you are running just to keep up!
The years fly by, and one fine day, business comes back and tells you - "the product is not good enough, and engineering does not seem to be able to give us what we need in time". What?! Are we talking about the same product that was beating the charts 5 years ago? Yes, we are. Unfortunately, while you were busy fixing bugs, adding new features, improving performance to meet the increasing load on the system, and fighting off impractical feature requests, the world has moved on. Competitors have come out with cooler stuff built on newer technology. Their solutions are more modular and can integrate with other services. They are more nimble and agile. On the other hand, your technology, that was shining new at inception is now rusty, your architecture looks dated and monolithic, your interfaces are not open enough. And guess what, over all those years, as you were madly keeping up with "Business as usual", technical debt has been silently creeping up behind you. The trickle of technical debt, that you always planned to catch up with in the next release, is now a mountain, blocking your way to agility, nimbleness and efficiency. Each new feature now takes longer to develop, and is costlier. No wonder business is complaining!
I see this story repeated again and again.
So, what is the solution?
Well, once you get to this state, there is no easy way out. So, my suggestion is, never let yourself get to this stage. Keep "watering the roots" - keep looking at ways to improve the architecture, keep refactoring and catching up with tech debt, keep an eye out for new technologies and trends and adopt what is necessary, keep in tune with business strategy and align the product roadmap accordingly, and of, course, use an Agile or Lean development methodology. These would help, but would not insulate you completely. You will still have challenges. But just being aware of the paradox and taking adequate steps should make life much easier.
