Lead Software Engineer (DSE)

at  EPAM Systems Inc

As a Lead Data Engineer, you will play a leading role in designing, building, and optimizing our data infrastructure, ensuring that it supports the advanced analytics need of the bank. You will oversee a team of data engineers, working closely with data analysts, DevOps team, infrastructure engineers, and other stakeholders to deliver high-quality data solution.

REQUIREMENTS

• Bachelor’s degree in Computer Science, Data Engineering, Information Technology or a related field
• 10+ years of experience as a Data Engineer with expertise in Hadoop, Spark and related technologies (HDFS, Hive, Ranger)
• Strong proficiency in Spark, Scala, and Python for data engineering tasks
• Experience with containerization (Docker, Kubernetes) and orchestration platforms like OpenShift
• Knowledge of DevOps practices, CI/CD pipelines and automation tools (Jenkins, Ansible, BitBucket)
• Experience with monitoring and logging tools (Grafana, Prometheus, Splunk) is a plus
• Strong problem-solving, troubleshooting, and communication skills
• Ability to work effectively in a fast-paced, collaborative environment
• Experience with cloud platforms (AWS, Azure, GCP) is a plus

WE OFFER

• By choosing EPAM, you’re getting a job at one of the most loved workplaces according to Newsweek 2021 & 2022&2023.
• Employee ideas are the main driver of our business. We have a very supportive environment where your voice matters
• You will be challenged while working side-by-side with the best talent globally. We work with top-notch technologies, constantly seeking new industry trends and best practices
• We offer a transparent career path and an individual roadmap to engineer your future & accelerate your journey
• At EPAM, you can find vast opportunities for self-development: online courses and libraries, mentoring programs, partial grants of certification, and experience exchange with colleagues around the world. You will learn, contribute, and grow with us

• Lead the design, development and implementation of high-performance Spark Scala applications and data processing pipelines for large-scale data
• Integrate Elasticsearch with Spark for efficient data indexing, querying and retrieval
• Optimize Spark jobs for performance and scalability on OpenShift Container Platform (OCP)
• Collaborate with cross-functional teams to translate data requirements into robust and scalable solutions
• Implement and maintain CI/CD pipelines for data engineering workflows, ensuring platform stability and efficient resource utilization
• Monitor and optimize data pipeline performance, troubleshoot issues and implement necessary enhancements
• Establish and maintain data quality and integrity throughout the data processing lifecycle
• Document data engineering processes, workflows and infrastructure configurations
• Stay abreast of emerging technologies and best practices in data engineering and DevOps
• Mentor and guide junior team members, fostering a culture of continuous learning and innovation