As a Senior Machine learning Quality Assurance Lead you will design, build, test, and deploy effective test automation solutions across a portfolio of different development projects. You will automate the testing effort to meet the short- and long-term goals of the company’s systems development by applying appropriate automation technologies. The ideal candidate will have experience in designing, developing, validating, and executing automated testing for artificial intelligence, machine learning, LLMs and the applications including web-based applications that incorporate these technologies. Eagerness to quickly learn and understand an evolving field is required, as there will be many opportunities to apply new skills in emerging areas of AI/ML development.Job Responsibilities

• Develops software that processes, stores and serves data and machine learning models for use by others.
• Develops moderate to large scale data structures and pipelines to organize, collect and standardize data that helps generate insights and intelligence to support business needs.
• Writes ETL (Extract / Transform / Load) or ELT processes, designs data stores and develops tools for real-time and offline analytic processing on premise or on cloud infrastructure.
• Develops and maintains optimal data pipelines into the ML and advanced analytics platform, including design of data flows, procedures, and schedules. Ensures that optimal data pipelines are scalable, repeatable and secure.
• Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of sources, assuring that they adhere to data quality and accessibility standards.
• Anticipates and prevents problems and roadblocks before they occur.
• Interacts with internal and external peers and managers to exchange complex information related to areas of specialization.
• Collaborates with AI/ML scientists and data scientists to prepare data for model development, and to deploy models to production.

BASIC QUALIFICATIONS

• Bachelor’s degree and at least 4 years of experience in machine learning, software engineering, or data engineering
• At least 3 years of experience working with SQL
• At least 3 years of experience programming in one or more of the following: Python, C, C++, Spark, Scala, and/or Java
• Experience establishing and maintaining key relationships with internal (peers, business partners and leadership) and external (business community, clients and vendors) within a matrix organization to ensure quality standards for service.
• Experience diagnosing, isolating, and resolving complex business issues and recommending and implementing strategies to resolve problems.
• At least 2 years of experience contributing to financial decisions in the workplace.
• At least 2 years of direct leadership, indirect leadership and/or cross- functional team leadership.
• Willing to travel up to 10% of the time for business purposes (within state and out of state)

PREFERRED QUALIFICATIONS

• Graduate Degree in a technical discipline and at least 2 years of experience in machine learning, software engineering, or data engineering combined with strong understanding of SQA (software Quality Assurance) methodologies.
• Experience with developing AI software applications
• Define test strategy, create test plans, author test cases, perform test execution, capture and communicate test results to support ongoing AI/ML team releases.
• Conduct functional, performance, and exploratory testing utilizing test automation frameworks using Python and AI-driven technologies. Design generative AI solutions for test case automation, smart test selection, and failure diagnosis. Develop continuous improvement steps incorporating machine learning models for test failure prediction, prioritization, and optimization.
• Collaborate with engineers and product teams on quality strategies and success metrics.
• Define and implement QA processes to optimize and scale testing.
• Lead the improvement and implementation of best practices, methodologies, and tooling to meet quality coverage.
• Integrate testing into the CI/CD deployment pipeline.
• Clearly and precisely articulate test results to application developers and management.
• Debug and resolve complex QA issues, including AI/ML-related challenges.
• Collaborate with software engineers and application users during the product development lifecycle.
• Establish and demonstrate compliance with team best practices and security requirements.Mentor and guide Quality Assurance Engineers, Software Engineers and ML Engineers of all levels on QA processes, methodologies and best practices.Depending on the need the individual may be overseeing the QA validation or performing hands on validation steps.
• 
  
  
  
• Experience deploying ML models at scale in production use open source tools (Kubeflow, Seldon, etc)

• Experience with DAG-based workflow orchestration systems (Airflow, Prefect, etc).
• Experience with data processing pipelines using Apache Spark or Databricks
• Experience with ML registries (MLFlow, etc)
• Experience with CICD pipelines (Tekton, Azure DevOps, Github Actions, etc)
• Experience with using or configuring Kubernetes in a multi-tenant environment.
• Experience troubleshooting distributed systems and reducing performance and costs
  We will consider employment of qualified applicants with arrest and conviction records

• Develops software that processes, stores and serves data and machine learning models for use by others.
• Develops moderate to large scale data structures and pipelines to organize, collect and standardize data that helps generate insights and intelligence to support business needs.
• Writes ETL (Extract / Transform / Load) or ELT processes, designs data stores and develops tools for real-time and offline analytic processing on premise or on cloud infrastructure.
• Develops and maintains optimal data pipelines into the ML and advanced analytics platform, including design of data flows, procedures, and schedules. Ensures that optimal data pipelines are scalable, repeatable and secure.
• Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of sources, assuring that they adhere to data quality and accessibility standards.
• Anticipates and prevents problems and roadblocks before they occur.
• Interacts with internal and external peers and managers to exchange complex information related to areas of specialization.
• Collaborates with AI/ML scientists and data scientists to prepare data for model development, and to deploy models to production