Optimization of bid request session storage

Problem:

The client used many Redis instances to store bid requests sessions for a short time. However, the number of Redis instances was growing rapidly, and a client would need more 120 Redis instances to handle the incoming traffic.

Solution:

We implemented a solution capable to handle 300K QPS and requires less than 5 servers and allows to store 25 times more sessions.
Developed Wrapper API on Node.js to integrate into existing infrastructure.
Developed CI/CD processes.

Timeframe:

1.5 week

Technology stack:
Golang