Software Engineer - Internal Observability (Lisbon)

at  DataDog

The Internal Observability SRE Team is part of the Observability SRE organization. Observability is a group in the SRE organization focused on managing Datadog’s internal observability tooling and practices, which includes using a variety of Datadog products. The group is focused on observability’s adoption and efficiency throughout Datadog’s organization, as it is a key aspect of an effective SRE approach for engineers to build and operate systems more effectively.
The Internal Observability SRE team specifically works on telemetry data (such as metrics, logs) collection, processing and usage. We integrate telemetry libraries into applications, define configuration for large-scale telemetry processing pipelines and focus on efficiency. Datadog has specific internal requirements (replication cross-region, resiliency) and we ensure Datadog can use Datadog reliably (even when Datadog has an incident). This work is done in partnership with the product teams behind these core Datadog products, and leads to product improvements for both Datadog itself and our customers.
The tools that we use daily are: Datadog, Bazel, Temporal, Terraform, Kubernetes. We work on Datadog’s main code repositories: Go, Python, Java.
At Datadog, we place value in our office culture - the relationships and collaboration it builds and the creativity it brings to the table. We operate as a hybrid workplace to ensure our Datadogs can create a work-life harmony that best fits them.

• Solve Datadog specific observability, instrumentation or telemetry collection problems, by designing and implementing internal tooling or contributing to our products.
• Work in partnership with our product teams to leverage new features and influence direction.
• Establish new golden paths for observability and build adoption for them across Engineering at Datadog (for eg. Telemetry ownership, Tracing, Multi-Regional Datadog Replication).
• Gather requirements for operational usecases with the relevant stakeholders (on-call/incident response, security investigations and incidents, compliance, …) and work on implementing relevant supporting telemetry and configuration. Our shared goal is a stronger resiliency and security posture.