Don't Make Monolith and Microservice your Religion (Amazon Prime Case)
Recently, we have been hearing a lot of articles discussing Amazon's Prime, Microservices, and Monolith. I didn't want to write this article, but I have noticed that many people, including developers, have missed the main point and come to binary opinions that mostly glorify the monolith while trashing microservices. This is not the main point.
If you are not aware of this commotion, Amazon Prime has published an article explaining how they reduced costs by 90% by migrating from microservices to a monolith. You can check the link below for the full article.
Of course, this has caused a commotion, since microservices are a well-known architecture for their high scalability. However, if you read the full article, you will see that it is not solely about microservices and monoliths. In this article, I would like to clarify and share my thoughts on this case.
Disclaimer: I am neither a supporter of monoliths nor microservices. Actually, I use both of them depending on the project.
Microservice is not the problem
In this use case, the root cause of the problem is not because they implemented it using microservices. To understand more, let's take a look at the old architecture.
The first problem is clear - it is the cost problem that occurred because AWS Step Function charges the user for each state transition, while in Amazon Prime's case, they had a lot of state transitions.
The second problem arose as a byproduct of choosing AWS Step Function for the detector service. They needed a way for the Media Conversion Service and Detector Service to communicate, and with Step Function, the methods for communication were limited. It's not surprising that they chose S3. However, using S3 made the cost and time bigger since they needed to upload and download large files.
So, we can conclude that the root cause was choosing the wrong technology (Step Function) that did not suit the use case. In the new architecture, they got rid of AWS Step Function. I guess they chose the new architecture because it was the easiest and fastest way to fix the issue. Let's take a look at the new architecture.
With this architecture, there is still a potential problem in the future. What if the needs of the detector function grow? Right now, the only option is to scale vertically and do a workaround by cloning the services, which is not efficient because it includes the media converter function, while what they need is just the detector function. See the picture below.
This is My Prediction
Later when scaling the detector function become annoying, they will create a microservice called Detector Service, but of course with another technology than Step Function
It’s Still Microservice
Let's take a look at the microservice schema above. So, there are User Service, User Address Service, and User Score Service. Someday, the engineers feel like the User, Address, and User Score services are in the same domain and decide to combine the three services into one service called it just User Service.
Does this make the architecture a monolith?
The same case happened in Amazon Prime. In my opinion, what they are doing is not dissecting the microservice. I believe that they still have Video Service, User Service, and so on. What they are doing is just consolidating all the distributed functions into one microservice (just call it Monitoring Service). So, if there are articles claiming that Amazon is dissecting microservices, I think it's just clickbait.
Conclusion
So the conclusions are:
The Amazon Prime case is not about Microservices → Monolith, but rather about Microservices → Another Microservice.
The real problem lies in choosing the wrong technology.
Read the article carefully and don't make engineering concepts such as Microservices or Monoliths your religion. Each architecture has its own cases, advantages, and disadvantages.
To close this, don't judge why they decided to make the initial architecture like that. I believe that there were a lot of variables when they decided to use Microservices with Step Functions. Actually, they could release the product and works, good job. We have to understand that every code that you write today will someday be rewritten, refactored, or no longer relevant.
Thank you for reading today's newsletter! If you find it valuable, here are some actions you can take:
1) ✉️ Subscribe — if you aren’t already, consider becoming a paid subscriber.
2) ❤️ Share — you can help spread the word by sharing the article with your team or someone who might find it useful!