[ Кейс ]
Масштабирование испанского приложения для борьбы с COVID:
интеграция двух потоков данных для 1 млн пользователей
Запрос клиента
The government of a European country commissioned a mobile app to estimate the likelihood of Covid-19 infection based on the phone owner's geolocation history.
Цель проекта
Our primary objective was to implement data streaming, ensuring it scales efficiently from a few users to millions.
Основные проблемы, с которыми мы столкнулись:
1. The complexity of this project was the need to merge multiple data streams. This required the use of an experimental tool, which we had no experience with yet.
2. Addressing unforeseen load testing challenges and infrastructure costs.
Решения и технологии
We worked with Python, Apache Beam, Airflow, and various Google Cloud services including Storage, BigQuery, Cloud SQL, Pub/Sub, Dataflow, and Composer.
The main challenge was integrating two data streams using Tula, a tool we hadn't previously used. The solution was designed to handle varying user loads, ranging from 10 users to 1 million.
Итог
.png)
We successfully developed a functional version of the application, aligning with the project's main objectives.
Dual data stream integration for