Turn raw market noise into Europe’s most trusted curated food‑industry market intelligence insights
A-INSIGHTS turns raw market signals into clear, actionable insights for decision-makers at names like McCain, FrieslandCampina and Ahold Delhaize. Our data pipelines are the first mile of that journey. As our next Medior Data Engineer, you’ll own a portfolio of connectors and ingestion services, designing resilient scrapers and API connectors that keep fresh, high-quality data flowing 24/7, while mentoring juniors and aligning engineering work with business goals.
‍

You are co-responsible for the front door of our central data platform

• Lead design & delivery of Python ingestion jobs (scrapers, API connectors) with robust error handling, idempotency and documented backfill strategies.
• Define the first steps of ELT: validation, normalization and schema mapping and agree on data contracts with downstream consumers.
• Own observability & reliability: define Service Level Indicators and Objectives, add logs/metrics/alerts, drive and track error budgets.
• Set standards & mentor: create connector templates/runbooks, lead code reviews, pair with juniors, and coach on clean code and debugging.
• Design for ethics & resilience: handle pagination, throttling and anti-bot challenges respectfully; follow legal guidelines; secure secrets/PII via Azure Key Vault and least privilege access.
• Embed AI enrichment (Azure OpenAI & Cognitive Services) to classify, summarize and deduplicate content; set acceptance criteria for model outputs.

‍