Principal Software Engineer (R-16245)

at  Dun Bradstreet

The Role: As a Principal Engineer, you will be leading work in an agile team focused on the design, implementation, and continuous enhancement of our complex web data engineering projects. These projects include web data scraping/mining, ML scoring, data aggregation, and data mastering. This role demands a proactive approach to leveraging cloud-native services and automation opportunities to ensure the highest quality of data in the Dun & Bradstreet Data Supply Chain.

KEY REQUIREMENTS:

• 10+ years of professional experience in Python, with a demonstrated ability to implement efficient and maintainable code, TDD and CI/CD practices.
• 10+ years of experience with RDMS, preferably PostgreSQL or ClickHouse, possessing a strong foundation in relational databases, SQL, PL/SQL and database optimization.
• Expertise in multiple programming languages such as Java and Scala.
• Proficiency in constructing data pipelines and familiarity with ETL processes.
• Proficiency in building Web scraping applications, familiarity with Web scraping frameworks
• Strong experience with API development, web services, and integrating third-party APIs.
• Deep understanding of HTTP, REST APIs and network protocols.
• Experience with cloud platforms (preferably GCP), including IaC tools like Terraform and Kubernetes.
• Strong analytical and problem-solving abilities, with a track record of working independently and adapting to new technologies and environments.
• Excellent communication skills, with an emphasis on writing clear technical documentation and collaborating effectively with both technical and non-technical teams.
• Bachelor’s or master’s degree in computer science, Engineering, or a related field, or equivalent professional experience.

• Lead the development and operational management of software and data platforms, ensuring adherence to the software development lifecycle and data management best practices.
• Design and manage robust data pipelines using Python, with tools like Dagster and Airflow for orchestration.
• Develop and enhance data ingestion and extraction systems, including API integrations and batch processing.
• Design and improve algorithms for data mastering, data matching, and data integrity to ensure the highest quality and accuracy of our datasets.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality data engineering solutions.
• Communicate effectively with customer success and other teams to ensure smooth operations and resolve any data-related issues.
• Drive business decision-making processes by creating actionable insights from aggregated datasets.
• Craft and refine database queries and visualization tools to monitor platform performance and identify potential issues.
• Continuously improve processes and tools to enhance data quality and pipeline efficiency.