Data Engineer - Stockholm

12 Apr, 2021 to 14 May, 2021

Develop and improve a data pipeline for a Machine Learning solution being deployed for XXX

The project is done together with a partner (responsible for mobile development and ingestion of data).

We have a preliminary pipeline in place where images are read from Azure Filestorage, fed to a system deployed with Docker-compose. The resulting data is persisted on Azure Cosmos DB.

This pipeline needs to be improved to allow for better scaling and guarantee availability.

Integration of the pipeline with other services running on Azure might be developed.

English OK


- Solid Azure experience (Azure SQL, Blob, Filestorage, Cosmos DB, ML Datasets)

- Docker, Docker-compose

- Python