[ Кейс ]
Обновление хранилища данных: в 8 раз более быстрые обновления для DAO Contributor Insights при практически нулевых затратах
.png)
.png)
Запрос клиента
The client, committed to scoring DAO contributors, faced mounting expenses due to repeated calls to third-party APIs. They sought to develop an external data warehouse to centralize and streamline information about contributors, ensuring convenient backend access.
Цель проекта
Our mission was to transition from weekly data updates to daily updates, significantly enhancing the update speed and drastically cutting down operational costs.
Решения и технологии
We utilized a combination of Python, Airflow, AWS Athena, dbt, AWS S3, and AWS Glue Data Studio to design and implement the desired data warehouse. This tech stack not only allowed us to accumulate and manage contributor data efficiently but also enabled swift and cost-effective updates.
Итог
A standout achievement of this project was the substantial cost-saving aspect; post the initial setup and migration, the cost of ownership became virtually negligible for the client.
Post-implementation, we successfully transitioned to daily data updates, accelerating the update process by a factor of