We are looking for a Data Engineer with strong backend engineering fundamentals and hands‑on experience building scalable, high‑performance data pipelines. The ideal candidate comes from a Java or Python backend background and has worked extensively with Apache Spark. Experience with Scala is a plus but not mandatory.
This role involves designing, developing, and optimizing data processing workflows, ensuring reliability, performance, and maintainability across distributed systems.
- Key ResponsibilitiesDesign and develop distributed data processing pipelines using Apache Spark.
- Build and maintain high‑performance backend services for data ingestion, transformation, and storage.
- Work with large‑scale datasets and optimize jobs for performance, scalability, and cost efficiency.
- Collaborate with data architects, analysts, and platform teams to deliver robust data solutions.
- Implement best practices for code quality, testing, CI/CD, and version control.
- Troubleshoot production issues and improve system reliability.
- Contribute to the design of data models, schemas, and storage strategies.
- Ensure data quality, lineage, and governance across pipelines.
- Required Skills & Experience3–5 years of experience as a Data Engineer or Backend Engineer.
- Strong programming skills in Java or Python (Scala optional).
- Hands‑on experience with Apache Spark (batch or streaming).
- Solid understanding of backend concepts:
- Multithreading & concurrency
- REST APIs / microservices
- Data structures & algorithms
- Performance optimization
- Experience with distributed systems and cloud‑based data platforms (AWS, Azure, or GCP).
- Familiarity with SQL and relational databases.
- Experience with version control (Git) and CI/CD pipelines.
- Good to HaveExperience with Scala for Spark development.
- Knowledge of Delta Lake, Iceberg, or Hive.
- Exposure to Kafka, event streaming, or real‑time data pipelines.
- Experience with containerization (Docker, Kubernetes).
- Understanding of data warehousing concepts.