Senior Data Engineer and Team Leader

at  International Data Group Inc

Overview:
Join a team that is building a leading intelligence platform for IT vendors!
IDC seeks a skilled, motivated, and dedicated Senior Data Engineer and Team Leader to help build some of the most advanced sophisticated data products. You will design and launch innovative analytical products through data operations in this role. You can showcase your industry-leading data design skills, work within a standard product development lifecycle, and collaborate with multiple development teams.
Project: Centralized Company List
Our mandate is to take the output of many different IDC research products, all of whom have or use company profile-related data, and combine that data into a centrally-managed database. Our goal is that the output will become the standard for company profile data across IDC, to be used across both existing and new research products. The role of the senior technical lead will be to take and develop requirements from stakeholder groups, design solution documents, and break them down into smaller pieces. Reporting to a business lead, you will oversee and support the technical team as a manager, as well contribute in executing more complex tasks. You will lead the development and continuous improvement of the company’s data architecture, ensuring the scalability, performance, and reliability of our company profile data systems.

QUALIFICATIONS

• BA/BS in Computer Science, Mathematics, Statistics, Data Engineering or 3-5 years of experience in a role requiring the application of these skills
• 3-5 years of relevant work experience in a Technical role
• Experience managing individuals for team outcomes
• Proficiency with Python scripts
• Proficiency with SQL and any major platforms, preferably Snowflake or PostgreSQL; alternatively, experience with any other database such as MySQL, Oracle, or Microsoft SQL.

Nice to haves:

• Familiar with Machine Learning, Gen AI
• Familiar with AWS stack or any other cloud environments

• Working with other data engineers from stakeholder groups to design and implement data models, storage solutions, and data integration processes.
• Enriching the dataset with data from third-party sources, webscraped information, and via AI
• Creating shared script packages, tools, APIs, and UIs for the input, quality control, and management of company profile data