Senior Data Engineer at Microsoft

Redmond, Washington, United States -

Full Time

Start Date

Immediate

Expiry Date

30 Mar, 26

Salary

234700.0

Posted On

30 Dec, 25

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

Skills

Data Engineering, Data Science, Machine Learning, Big Data, Azure Services, SQL, Python, C#, Data Pipelines, Data Modeling, Artificial Intelligence, Data Quality, Statistical Analysis, Data Visualization, ETL, Hadoop Ecosystem, Apache Spark

Industry

Software Development

Description

Overview Microsoft Cloud Operations + Innovation (CO+I) is the engine that powers Microsoft's cloud services and our team is focused on delivering high quality infrastructure to support cloud operations. As Microsoft’s Cloud business continues to mature, our infrastructure expansion accelerates, and Data Centers are central to this growth. To support this, the acquisition and development of our owned, designed, and constructed Data Center facilities will scale to meet the demands of our customers, while we also continue to lease and acquire Data Center capacity at pace, particularly in our high growth markets, working closely with Data Center operators in each region, and across the Globe. We are seeking a skilled Data Scientist to join our COI+I Lease and Land Development Digital Transformation team. This role is ideal for someone who thrives in a fast-paced environment, enjoys solving complex problems, and is passionate about using data to influence product strategy and customer experience. You will work closely with cross-functional teams including PMs, engineers, and business stakeholders to deliver AI/Agentic AI Solutions and actionable insights and scalable data solutions. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day Responsibilities Apply modification techniques to transform raw data into compatible formats for downstream systems. Utilize software and computing tools to ensure data quality and completeness. Implement code to extract and validate raw data from upstream sources, ensuring accuracy and reliability. Writes efficient, readable, extensible code from scratch that spans multiple features/solutions. Develops technical expertise in proper modeling, coding, and/or debugging techniques such as locating, isolating, and resolving errors and/or defects. Leverages technical proficiency of big-data software engineering concepts, such as Hadoop Ecosystem, Apache Spark, continuous integration and continuous delivery (CI/CD), Docker, Delta Lake, MLflow, AML, and representational state transfer (REST) application programming interface (API) consumption/development Acquires data necessary for successful completion of the project plan. Proactively detects changes and communicates to senior leaders. Develops usable data sets for modeling purposes. Contributes to ethics and privacy policies related to collecting and preparing data by providing updates and suggestions around internal best practices. Contributes to data integrity/cleanliness conversations with customers Adhere to data modeling and handling procedures to maintain compliance with laws and policies. Document data type, classifications, and lineage to ensure traceability and govern data accessibility. Perform root cause analysis to identify and resolve anomalies. Implement performance monitoring protocols and build visualizations to monitor data quality and pipeline health. Support and monitor data platforms to ensure optimal performance and compliance with service level agreements. Knowledge and implementation of an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Leverages knowledge of machine learning solutions (e.g., classification, regression, clustering, forecasting, NLP, image recognition, etc.) and individual algorithms (e.g., linear and logistic regression, k-means, gradient boosting, autoregressive integrated moving average [ARIMA], recurrent neutral networks [RNN], long short-term memory [LSTM] networks) to identify the best approach to complete objectives. Understands modeling techniques (e.g., dimensionality reduction, cross validation, regularization, encoding, assembling, activation functions) and selects the correct approach to prepare data, train and optimize the model, and evaluate the output for statistical and business significance. Understands the risks of data leakage, the bias/variance tradeoff, methodological limitations, etc. Writes all necessary scripts in the appropriate language: T-SQL, U-SQL, KQL, Python, R, etc. Constructs hypotheses, designs controlled experiments, analyzes results using statistical tests, and communicates findings to business stakeholders. Effectively communicates with diverse audiences on data quality issues and initiatives. Understands operational considerations of model deployment, such as performance, scalability, monitoring, maintenance, integration into engineering production system, stability. Develops operational models that run at scale through partnership with data engineering teams. Coaches less experienced engineers on data analysis and modeling best practices. Develops a strong understanding of the Microsoft toolset in artificial intelligence (AI) and machine learning (ML) (e.g., Azure Machine Learning, Azure Cognitive Services, Azure Databricks). Design and Implement Dashboards: Develop user-friendly dashboards for various applications, such as Supplier Spend Analytics, Supplier Scorecards, Incident and Service Level Agreement (SLA) Compliance Monitoring, Spares and Inventory Management, and other business-facing applications. Qualifications Required qualifications Bachelor’s degree in computer science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years’ experience in business analytics, data science, data modeling, or data engineering work OR master’s degree in computer science, Math, Software Engineering, Computer Engineering, or related field and 3+ years’ experience in business analytics, data science, data modeling, or data engineering work. Background Check Requirements: Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter. Preferred qualifications include: 8+ years of experience in data engineering with coding and debugging skills in C#, Python, and/or SQL Deploying solutions in Azure Services & Managing Azure Subscriptions Understanding and knowledge about big data and writing queries with Kusto/KQL. Understanding and knowledge about extracting data via REST APIs Strong analytical skills with a systematic and structured approach to software design Strong analytical skills with a systematic and structured approach to software design 5+ years of experience in data science, analytics, or machine learning 4+ years of experience in developing solutions with Microsoft Power Platform, including Power BI, Fabric, Power Automate & M365 Dataverse. 3+ years of experience in building Data Pipelines using Azure Data Factory. 1+ year of experience in developing solutions in Azure Fabric 4+ Years of experience in writing SQL Queries Experience with data cloud computing technologies such as – Azure Synapse, Azure Data Factory, SQL, Azure Data Explorer 5+ years of experience in Microsoft/Azure Data Stack including , ETL, Data Pipeline development with SQL, Fabric Data Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $158,400 - $258,000 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.

Responsibilities

The Senior Data Engineer will transform raw data into compatible formats, ensure data quality, and develop scalable data solutions. They will also implement AI solutions and collaborate with cross-functional teams to deliver actionable insights.