Senior Data Engineer (with Backend Experience) at CodiLime

Warsaw, Masovian Voivodeship, Poland -

Full Time

Start Date

Immediate

Expiry Date

02 Jul, 26

Salary

0.0

Posted On

03 Apr, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Python, SQL, Snowflake, dbt, Apache Airflow, Apache Spark, AWS, Azure, FastAPI, ETL pipelines, Data modeling, Git, Object-oriented programming, CI/CD, Data quality, API development

Industry

IT Services and IT Consulting

Description

Get to know us better CodiLime is a software and network engineering industry expert and the first-choice service partner for top global networking hardware providers, software providers and telecoms. We create proofs-of-concept, help our clients build new products, nurture existing ones and provide services in production environments. Our clients include both tech startups and big players in various industries and geographic locations (US, Japan, Israel, Europe). While no longer a startup - we have 250+ people on board and have been operating since 2011 we’ve kept our people-oriented culture. Our values are simple: Act to deliver. Disrupt to grow. Team up to win. The project and the team You will work on a large-scale data platform built for a global consulting firm, supporting M&A (Mergers and Acquisitions) analysis. The project consists of: A cloud-based platform for data visualization (APIs and services) A large, integrated dataset (300M+ company records from 10+ sources) Your primary focus will be on data engineering (approx. 80%), including building and improving data pipelines and dataset quality. Additionally, you will contribute to backend and platform development (approx. 20%), including features, APIs, and integration with the data layer. Technology stack: Languages: Python, SQL Data: Snowflake, dbt Workflow Orchestration: Apache Airflow (complex DAGs) Data Processing: Apache Spark - Platform Cloud: AWS (EKS, S3, Lambda, ECR, EMR, Opensearch) - Platform Azure (AKS, Blob Storage, Azure Functions, ACR, Databricks, AI Search) - Dataset API: API Gateway, FastAPI (REST, async) CI/CD: GitHub Actions Data Scale: 300M+ records across multiple sources Future Direction: Agentic AI systems LangChain and cloud-native AI libraries and frameworks Azure OpenAI integration What else you should know: Team: Data Architecture Lead, Data Engineers, DataOps Engineers, Frontend Engineer, Backend Engineer, Product Owner Distributed team across Europe and India Agile, collaborative, proactive and impact-driven environment We work on multiple projects at the same time, so we may suggest a different project if it better matches your experience and profile. Your role This is a hybrid role combining Data Engineering and Backend Engineering, with a stronger focus on the data layer. You will: Build and maintain scalable ETL/data pipelines (Python, SQL, dbt, Snowflake, Airflow) Design and optimize data models, transformations and workflows Ensure data quality, consistency, and performance Integrate multiple data sources into a unified dataset Contribute to backend development (APIs, data access layer, backend-frontend contracts) Support platform integrations and cloud-based solutions Write clean, testable code and participate in code reviews Document data processes and system architecture Do we have a match? As a Senior Data Engineer, you must meet the following criteria: Strong experience with Snowflake and dbt Strong Python skills Ability to write clean, scalable, and testable code (including unit tests) Solid understanding and applying object-oriented programming (OOP) Experience with version control systems: Git Experience with AWS or Azure Good knowledge of English (minimum C1 level) Beyond the criteria above, we would appreciate the nice-to-haves: Experience with Apache Spark (preferably Databricks) Experience with Azure AI Search or AWS OpenSearch Experience designing ETL/ELT pipelines Familiarity with LLMs, Azure OpenAI, or agentic AI systems More reasons to join us Flexible working hours and approach to work: fully remotely, in the office or hybrid Professional growth supported by internal training sessions and a training budget Solid onboarding with a hands-on approach to give you an easy start A great atmosphere among professionals who are passionate about their work

Responsibilities

You will build and maintain scalable ETL and data pipelines while ensuring data quality and performance. Additionally, you will contribute to backend development by designing APIs and integrating data layers into the platform.