Sr. Director, Responsible AI Testing, Evaluation, and Alignment

at  Salesforce

To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts.
Job Category
Data
Job Details

MINIMUM REQUIREMENTS:

• A related technical degree required
• 8+ years of relevant technical experience in Software Engineering Management, AI ethics, AI research, Data Science Management, or similar roles - of which, at least 2+ years are in a relevant leadership/people management role
• Proven leadership, organizational, and execution skills. Passion for developing pioneering AI ethics technology and deploying it through a multi-stakeholder approach
• Experience in applying ML and AI technologies in a Generative AI context responsibility/ethically; solid ML-focused engineering and research skills, particularly around using and training models and deploying them in systems
• Have proficiency in SQL, Python, and data analysis/data mining tools.
• Experience working in a technical environment with a broad, cross-functional team to get results, define requirements, coordinate resources from other groups, and guide the team through key landmarks
• Excellent written and oral communication skills, as well as interpersonal skills, including the ability to articulate technical concepts to both technical and non-technical audiences

PREFERRED REQUIREMENTS:

• 10+ years of relevant experience in AI ethics, AI research, Software Engineering Management, Data Science Management, Trust & Safety, or similar roles
• Advanced degree in Computer Science, Engineering, Information Systems, Data Science, Statistics, or a related field
• Proven track record of influential projects and publications in relevant fields and top-tier conferences and journals (such as NeurIPS, ICML, AAAI, CHI, FAccT).
  In office expectations are 36 days/a quarter to support customers and/or collaborate with their teams.
  Accommodations
  If you require assistance due to a disability applying for open positions please submit a request via this Accommodations Request Form .
  Posting Statement
  At Salesforce we believe that the business of business is to improve the state of our world. Each of us has a responsibility to drive Equality in our communities and workplaces. We are committed to creating a workforce that reflects society through inclusive programs and initiatives such as equal pay, employee resource groups, inclusive benefits, and more. Learn more about Equality at www.equality.com and explore our company benefits at www.salesforcebenefits.com .
  Salesforce is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Salesforce does not accept unsolicited headhunter and agency resumes. Salesforce will not pay any third-party agency or company that does not have a signed agreement with Salesforce .
  Salesforce welcomes all.
  Pursuant to the San Francisco Fair Chance Ordinance and the Los Angeles Fair Chance Initiative for Hiring, Salesforce will consider for employment qualified applicants with arrest and conviction records.
  For New York-based roles, the base salary hiring range for this position is $224,100 to $372,900.

JOB RESPONSIBILITIES:

Develop and run the Responsible AI team’s roadmap of contributions toward Salesforce’s overall Trusted AI/ML lifecycle, ensuring we are safely training, aligning, evaluating, implementing mitigations, deploying, and monitoring models. With AI ethicists and AI researchers, ensure the responsible and safe development, evaluation, and maintenance of generative foundation, and predictive models.
Drive the workstream to identify, set, and evaluate standards of ML/AI models in collaboration with AI ethicists and AI research scientists; support monitoring them post-deployment for continual improvement.

Run ongoing ethical evaluation and testing (including adversarial testing and red teaming) of ML/AI models and products:

• With the team, implement automation strategies for conducting AI Red Teaming and Testing and the development of tools, data, and pipelines to support that work
• With the team, build tools to enable testing to take place at scale. Enable product/engineering teams to perform their own tests through tooling and playbooks.

Lead the process of scoping, documenting, and performing testing with partner teams, including the implementation of mitigations identified during testing.
Lead development and testing of content safety strategy, including culturally critical localization and globalization efforts.
In collaboration with others, help establish mechanisms for ongoing monitoring, evaluation, and reporting of AI systems to ensure alignment with ethical standards and regulatory requirements.
Lead the creation of evaluation datasets for potential risks and harms of language and multimodal models.
As a player-coach, provide guidance, mentorship, and technical leadership to a multidisciplinary team of technical and socio-technical authorities.
As part of the Responsible AI leadership team, create the technical vision and implement the strategic roadmap to advance AI trust and safety at Salesforce.