Data Engineer - Stockholm

Develop and improve a data pipeline for a Machine Learning solution being deployed for XXX

The project is done together with a partner (responsible for mobile development and ingestion of data).

We have a preliminary pipeline in place where images are read from Azure Filestorage, fed to a system deployed with Docker-compose. The resulting data is persisted on Azure Cosmos DB.

This pipeline needs to be improved to allow for better scaling and guarantee availability.

Integration of the pipeline with other services running on Azure might be developed.

English OK

Requirements:

- Solid Azure experience (Azure SQL, Blob, Filestorage, Cosmos DB, ML Datasets)

- Docker, Docker-compose

- Python