Primary Talent Partners has a new contract opening for a Lead Data Engineer with our large power and utilities client in Charlotte, NC. This is a 12-month contract with a potential for extension.
Pay: $92.25 - $102.25/hr; W2 contract, no PTO, no Benefits. ACA-compliant supplemental package available for enrollment.

Description:

• Support or collaborate with application developers, database architects, data analysts and data scientists to ensure optimal data delivery architecture throughout ongoing projects/operations.
• Design, build, and manage analytics infrastructure that can be utilized by data analysts, data scientists, and non-technical data consumers, which enables functions of the big data platform for Analytics.
• Develop, construct, test, and maintain architectures, such as databases and large-scale processing systems that help analyze and process data in the way the Analytics organization requires.
• Develop highly scalable data management interfaces, as well as software components by employing programming languages and tools.
• Work closely with a team of Data Science staff to take existing or new models and convert them into scalable analytical solutions.
• Design, document, build, test and deploy data pipelines that assemble large complex datasets from various sources and integrate them into a unified view.
• Identify, design, and implement operational improvements: automating manual processes, data quality checks, error handling and recovery, re-designing infrastructure as needed.
• Create data models that will allow analytics and business teams to derive insights about customer behaviors
• Build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications.
• Responsible for obtaining data from the System of Record and establishing batch or real-time data feed to provide analysis in an automated fashion.
• Develop techniques supporting trending and analytic decision making processes
• Apply technologies for responsive front-end experience
• Ensure systems meet business requirements and industry practices
• Research opportunities for data acquisition and new uses for existing data
• Develop data set processes for data modeling, mining and production
• Integrate data management technologies and software engineering tools into existing structures
• Employ a variety of languages and tools (e.g. scripting languages)

Core Responsibilities:

• Provides technical direction, guides the team on key technical aspects and responsible for product tech delivery
• Lead the Design, Build, Test and Deployment of components
• Where applicable in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead)
• Understand requirements / use case to outline technical scope and lead delivery of technical solution
• Confirm required developers and skillsets specific to product
• Provides leadership, direction, peer review and accountability to developers on the product (key responsibility)
• Works closely with the Product Owner to align on delivery goals and timing
• Assists Product Owner with prioritizing and managing team backlog
• Collaborates with Data and Solution architects on key technical decisions
• The architecture and design to deliver the requirements and functionality

Core Experience and Abilities:

• Power Systems engineering experience
• DER Dispatch experience
• Automated fault analysis systems experience
• Ability to perform hands on development and peer review for certain components / tech stack on the product
• Standing up of development instances and migration path (with required security, access/roles)
• Develop components and related processes (e.g. data pipelines and associated ETL processes, workflows)
• Lead implementation of integrated data quality framework
• Ensures optimal framework design and load testing scope to optimize performance (specifically for Big Data)
• Supports data scientist with test and validation of models
• Performs impact analysis and identifies risk to design changes
• Ability to build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications
• Ensures Test Driven development
• Experience leading teams to deliver complex products
• Strong technical skills and communication skills
• Strong skills with business stakeholder interactions
• Strong solutioning and architecture skills
• Experience building real time data ingestion streams (event driven)
• Ensure data security and permissions solutions, including data encryption, user access controls and logging

Core Technical Skills:

• Experience with native AWS technologies for data and analytics such as Athena, S3, Lambda, Glue, EMR, Kinesis, SNS, CloudWatch, etc.
• Tools and Languages such as Django, Python, Java, Scala, Pandas
• Infrastructure as Code technology such as Terraform
• AWS services such as S3, EMR, Glue, Lambda, Athena, Kinesis, EC2, SNS, SQS, Cloudwatch
• Experience with databases such as Redshift, Document DB, DynamoDB and Mongo DB
• Experience transitioning on premise big data platforms into cloud based platforms such as AWS

Additional Technical Skills (nice to have, but not required for the role):

• Hadoop platform (Hive; HBase; Druid)
• Spark
• PySpark

Core Responsibilities:

• Provides technical direction, guides the team on key technical aspects and responsible for product tech delivery
• Lead the Design, Build, Test and Deployment of components
• Where applicable in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead)
• Understand requirements / use case to outline technical scope and lead delivery of technical solution
• Confirm required developers and skillsets specific to product
• Provides leadership, direction, peer review and accountability to developers on the product (key responsibility)
• Works closely with the Product Owner to align on delivery goals and timing
• Assists Product Owner with prioritizing and managing team backlog
• Collaborates with Data and Solution architects on key technical decisions
• The architecture and design to deliver the requirements and functionalit

Additional Technical Skills (nice to have, but not required for the role):

• Hadoop platform (Hive; HBase; Druid)
• Spark
• PySpar