About the job

LLM Platform Engineer

Berlin, Germany — Hybrid

Developer Tools | AI SaaS | LLM Infrastructure | Platform Engineering

Our client, a growing AI SaaS / Developer Tools company based in Berlin, is looking for an LLM Platform Engineer to build the internal platform that enables engineering teams to deploy, evaluate, observe, and operate LLM-powered applications at scale.

You'll work across AI Engineering and Platform Engineering, creating the shared infrastructure behind model access, retrieval, evaluation, observability, and production deployment.

What You'll Work On

• Build platform services for production LLM applications

• Develop model gateways and APIs for accessing multiple LLM providers

• Build shared RAG and vector retrieval infrastructure

• Develop Python tooling and services for AI engineering teams

• Deploy and operate AI workloads on Kubernetes

• Build evaluation pipelines for LLM and RAG applications

• Implement observability across prompts, models, retrieval, latency, cost, and failures

• Automate infrastructure and environments using Terraform

• Create deployment tooling and self-service capabilities for engineering teams

• Improve reliability, scalability, and operational standards across production AI applications

Core Skills

• 4+ years in Platform Engineering, AI Engineering, MLOps, Backend Engineering, or similar roles

• Python

• Kubernetes

• RAG

• Vector Databases

• LLM APIs

• Observability

• Terraform

• Strong understanding of production cloud-native systems

Nice to Have

OpenAI, Anthropic, or multiple model providers

Pinecone, Weaviate, Qdrant, or pgvector

LangGraph / LangChain

LLM gateways or routing platforms

OpenTelemetry

LLM evaluation and tracing tools

AWS / GCP

GitOps / Argo CD

Model Context Protocol (MCP)

• Experience building internal developer platforms