Principal Data Engineer at ATPCO

Herndon, Virginia, United States -

Full Time

Start Date

Immediate

Expiry Date

06 Jan, 26

Salary

162431.0

Posted On

08 Oct, 25

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Data Engineering, Data Pipelines, ETL Processes, AWS Services, Data Warehousing, Data Lakes, Big Data Platforms, Real-Time Data Streaming, Data Security, Data Mesh Principles, Python, Java, Scala, SQL, Machine Learning, Data Modeling

Industry

Airlines and Aviation

Description

Company Description ATPCO is the foundation of flight shopping, providing pricing and retailing data, tools, and services to 500+ airlines, global distribution systems, sales channels, and technology companies. ATPCO links the entire airline community together, collaborating to develop industry standards for airline distribution and end-to-end technology solutions. From shopping to settlement, ATPCO solutions work seamlessly across existing, new, and evolving technologies and methods. Airline-owned and reliably supporting air travel for more than 55 years, ATPCO is everywhere people buy flights. We consider qualified applicants for employment without regard to race, gender, age, color, religion, national origin, citizenship status, marital status, disability, sexual orientation, protected military/veteran status, gender identity or expression, genetic information, marital status, medical condition, or any other legally protected factor. Job Description As a Principal Data Engineer, you will be responsible for building and optimizing data pipelines, managing data storage and processing systems, and ensuring the availability, scalability, and reliability of our data platform. You will collaborate with cross-functional teams, including data scientists, software engineers, and business stakeholders to understand data requirements and deliver efficient high-quality data solutions. We are seeking an accomplished individual to join our esteemed team of ATPCO data engineers, dedicated to tackling intricate data processing challenges within the Airline industry. This role offers an opportunity to work on cutting-edge solutions that involve handling hundreds of terabytes of data daily, leveraging the latest AWS services. You Will: Partner with data scientists, analysts, and cross-functional stakeholders to translate business and ML/AI use cases into scalable data architectures, including designing modern schemas, data models, and pipelines that enable advanced analytics, machine learning, and real-time reporting. Design, develop, and maintain scalable and efficient data pipelines and ETL processes to ingest, process, and transform large volumes of data from various sources into usable formats Build and optimize data storage and processing systems, including data warehouses, data lakes, and big data platforms, using AWS services such as Amazon Redshift, AWS Glue, AWS EMR, AWS S3, and AWS Lambda, to enable efficient data retrieval and analysis Implement and manage real-time data streaming architectures using AWS services like Amazon Kinesis or Apache Kafka to enable real-time data processing and analytics Ensure that solutions facilitate secure, efficient, and real-time data analysis and reporting, leveraging infrastructure-as-code and adopting best practices for automation, monitoring, cost optimization, and compliance across cloud environments. Perform data profiling, data cleansing, and data transformation tasks to prepare data for analysis and reporting Implement data security and privacy measures to protect sensitive and confidential data using AWS security services and features Design and implement data architectures following Data Mesh principles within the AWS environment, including domain-oriented data ownership, self-serve data infrastructure, and federated data governance Provide technical guidance and mentorship to junior data engineers, reviewing their work and ensuring adherence to best practices and standards Qualifications: Strong programming skills in languages like Python, Java, or Scala, with experience in data manipulation and transformation frameworks Proven experience as a data engineer, with experience in designing and building large-scale data processing systems Strong understanding of data modeling concepts and data management principles In-depth knowledge of SQL and experience working with relational and non-relational databases Knowledge of Data Mesh principles and experience designing and implementing data architectures following Data Mesh concepts within the AWS ecosystem Experience with real-time data streaming architectures using AWS services like Amazon Kinesis or Apache Kafka Familiarity with AWS cloud services, such as AWS Sagemaker Unified Studio, AWS Glue, AWS Lambda, AWS EMR, AWS S3, Amazon Redshift, and their data-related features and functionalities Familiarity with AWS security services and features for data security and privacy [Bonus] Experience designing, implementing, and managing scalable machine learning pipelines and MLOps frameworks for production AI/ML solutions in cloud environments (e.g., AWS, Azure, GCP), including model deployment, monitoring, and operational automation. Bachelor's or Master's degree in Computer Science, Information Systems, or a related field Salary Range: $145,290 – 162,431 *The disclosed range estimate has not been adjusted for applicable geographic differential associated with the location* Additional Information At ATPCO, we are deeply committed to diversity, equity, and inclusion. Our supportive policies promote work-life balance through flexible work arrangements, and we cultivate a workplace where every employee feels valued, respected, and a true sense of belonging. We consider qualified applicants for employment without regard to race, gender, age, color, religion, national origin, citizenship status, marital status, disability, sexual orientation, protected military/veteran status, gender identity or expression, genetic information, marital status, medical condition, or any other legally protected factor All your information will be kept confidential according to EEO guidelines.

Responsibilities

As a Principal Data Engineer, you will build and optimize data pipelines, manage data storage and processing systems, and ensure the reliability of our data platform. You will collaborate with cross-functional teams to deliver high-quality data solutions for the airline industry.