Global IT Data Engineer Senior Specialist - DaaS Lighthouse at boston consulting group

Gurgaon, haryana, India -

Full Time

Start Date

Immediate

Expiry Date

20 Jun, 26

Salary

0.0

Posted On

22 Mar, 26

Experience

10 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Data Integration, Pipeline Management, Snowflake, Airflow DAGs, Python, APIs, Streamlit, AWS S3, Google Cloud Storage, GenAI Workflows, Data Governance, dbt, Agile, Scrum, Kanban, SQL

Industry

Business Consulting and Services

Description

Who We Are Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we help clients with total transformation-inspiring complex change, enabling organizations to grow, building competitive advantage, and driving bottom-line impact. To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge management consulting along with technology and design, corporate and digital ventures—and business purpose. We work in a uniquely collaborative model across the firm and throughout all levels of the client organization, generating results that allow our clients to thrive. What You'll Do As a Data Engineer Senior Specialist, you will drive the integration, ingestion, and optimization of external data into BCG’s environment. This individual contributor role offers an opportunity to innovate, collaborate, and influence data engineering practices across BCG and externally. Core Responsibilities: Data Integration & Pipeline Management: · Lead the onboarding and integration of proprietary and third-party datasets into Snowflake and related applications. · Design, develop, document, and maintain robust data pipelines, ensuring seamless ingestion, migration, and rigorous testing. · Ensure correctness, completeness, and integrity of transformed data utilized in analytical dashboards. · Collaborate closely with case teams, product teams, knowledge teams, vendors, and clients to ensure alignment and secure stakeholder sign-off. Technical Operations & Innovation: · Develop and manage automated data workflows using Airflow DAGs, Snowflake, Python, APIs, schemas, Streamlit, and data storage solutions such as AWS S3 and Google Cloud Storage. · Continuously monitor data pipelines, proactively troubleshoot issues, and swiftly implement resolutions. · Evaluate emerging technologies and conduct MVPs or PoCs to introduce new data engineering capabilities. · Actively monitor and address technical challenges requiring immediate resolution to minimize disruption. · Design and implement MCP-based GenAI workflows and solutions to enhance data assets and products. Stakeholder & Vendor Collaboration: · Act as a key technical liaison between internal stakeholders and external data vendors. · Manage integration projects, facilitating data quality checks, ensuring compliance, and promptly addressing discrepancies. · Optimize processes and workflows based on stakeholder feedback and project outcomes. Data Governance & Documentation: · Maintain comprehensive metadata logs, asset templates, and technical documentation. · Ensure robust technical compliance and manage user access controls within the Lighthouse data ecosystem. · Implement data performance and security policies aligned with governance objectives and regulatory standards. What You'll Bring 8+ years of experience in Data Engineering, Data Integration, or related data-focused roles. Passion for solving complex, business-oriented data challenges end-to-end, demonstrating precision and attention to detail. Understanding of data strategy, management concepts, processes, and tools such as Data Catalog, Data Lineage, and Data Quality. Demonstrated proficiency with technologies including Snowflake, Python, Airflow, Streamlit, AWS S3, Google Cloud Storage, dbt, and Apache Airflow. Experience working in Agile environments using methodologies such as Scrum and Kanban. Strong customer orientation and proven ability to build effective working relationships, resolve conflicts, and deliver impactful outcomes. Demonstrate ability to write new code i.e., well-documented and stored in a version control system Experience working with agile methodologies (Scrum, Kanban) and Meta Scrum with cross-functional teams (Product Owners, Scrum Master, Architects, and data SMEs) Strong communication skills with proven experience in stakeholder engagement (including senior business and technical teams) and the ability to showcase the business benefits of Data Governance. Bachelor’s degree (or higher) in Computer Science, Engineering, Mathematics, Statistics, or a related discipline. Who You'll Work With BCG Global Consulting Teams: Functional & Industry practice areas, Data Product Portfolio, Portal/ BCG Marketplace, Master Data Management, and Data Governance teams BCG IT and Knowledge Teams: Business intelligence, analytics, operational, and technical specialists. Cross-Functional Teams: Data Protection office, Data Governance, Data Engineers, Knowledge Function, Enterprise Services, Data Owners, Stewards, IT, Data Ops/ Warehouse and Infrastructure. Data Layer Team: Data engineers, enterprise architects, product analysts, data stewards, and QA engineers, enhancing BCG’s data products and infrastructure. Additional info YOU’RE GOOD AT · Designing and building scalable, automated data ingestion and transformation pipelines (ETL/ELT). · Analyzing and troubleshooting data quality and performance bottlenecks, and providing recommendations for enhancements. · Strong analytical skills in handling structured, semi-structured, and unstructured datasets. · Collaborating effectively with product owners, stakeholders, and technical teams to define requirements, deliver trusted results, and foster innovation. · Building processes that manage data transformation, metadata, dependencies, and workload effectively. · Advanced SQL proficiency and deep experience with analytical data warehouses, particularly Snowflake. · Demonstrated capability in applying Slowly Changing Dimensions (SCD), Change Data Capture (CDC), and Data Quality/Data Validation (DQ/DV) frameworks. · Writing clean, efficient, and well-documented code stored within version control systems · Familiarity with tools and technologies including dbt, Apache Airflow, Python, Snowflake, and Streamlit, as well as JIRA and Confluence. · Proven knowledge on building data and AI solutions using MCP servers. · Keeping abreast of advancements in data engineering practices and technologies. · Working knowledge of ingesting and handling different types of source data effectively. · Familiarity with medallion architecture and data mesh principles (preferred). · Strong communication skills and proven ability to effectively engage stakeholders at various levels. Boston Consulting Group is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, age, religion, sex, sexual orientation, gender identity / expression, national origin, disability, protected veteran status, or any other characteristic protected under national, provincial, or local law, where applicable, and those with criminal histories will be considered in a manner consistent with applicable state and local laws. BCG is an E - Verify Employer. Click here for more information on E-Verify.

Responsibilities

The Senior Specialist will lead the integration, ingestion, and optimization of external data into BCG's environment, focusing on designing, developing, and maintaining robust data pipelines for ingestion and transformation into Snowflake. Key duties include developing automated workflows using tools like Airflow and Python, monitoring data quality, and collaborating with various internal and external stakeholders.