Lead Data Engineer at Dice
California, California, USA -
Full Time


Start Date

Immediate

Expiry Date

24 Dec, 26

Salary

120000.0

Posted On

14 Nov, 26

Experience

5 year(s) or above

Remote Job

Yes

Telecommute

Yes

Sponsor Visa

No

Skills

Industry

Information Technology & Services

Description

Job Details

Skills

  • SQL
  • Python
  • Scala
  • PySpark
  • Microsoft Azure
  • Extract, Transform, Load
  • Artificial Intelligence
  • Machine Learning Operations (ML Ops)
  • Apache Spark
  • Finance
  • Financial Services
  • ML

Summary

Job Description:


  1. Python, PySpark, Scala and SQL – strong production-level data engineering skills.
  2. Azure Databricks – Delta Lake, Spark performance tuning, Jobs/Workflows and Unity Catalog.
  3. Azure data services – ADLS Gen2 and Azure Data Factory (ADF).
  4. Lakehouse / Medallion Architecture – proven experience implementing Bronze, Silver and Gold layers at enterprise scale.
  5. Data Pipeline Engineering – metadata-driven ingestion frameworks, data transformation, data quality and large-scale pipeline optimization.
  6. Production AI/LLM Experience – hands-on experience designing and deploying LLM-based solutions, including RAG, embeddings, retrieval and prompt engineering.
  7. LLM Frameworks – experience with LangChain, LlamaIndex or LangGraph, plus Azure OpenAI, OpenAI or Databricks Model Serving.
  8. Azure DevOps / CI-CD – experience automating data pipeline deployments; Terraform and Databricks Asset Bundles are highly relevant.
  9. Technical Leadership – ability to design solutions, write production code, review code and guide data engineering teams.
  10. Financial services or insurance data, especially financial reconciliation, general ledger or accounting data.
  11. AI-driven data quality, anomaly detection and automated reconciliation.
  12. Knowledge graphs, ontologies, semantic layers or enterprise Text-to-SQL solutions.
  13. Azure security, data governance, PII protection and fine-grained access controls.
  14. MLOps/LLMOps, model evaluation and production AI monitoring.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

  • Dice Id: 91142017
  • Position Id: 9139951
  • Posted 2 days ago
  •  

How To Apply:

Incase you would like to apply to this job directly from the source, please click here

Responsibilities

Loading...