Staff Backend Software Engineer - Databases - Loki Ingest

About the Role

Join the Databases team at Grafana Labs to build and maintain critical open-source telemetry databases for metrics, logs, traces, and profiles. This role offers a unique opportunity to contribute to a global, remote-first company at the forefront of observability technology.

About the Team

The Databases team is responsible for critical open-source telemetry databases, including Mimir for metrics, Loki for logs, Tempo for traces, and Pyroscope for profiles. These are multi-tenant distributed systems built in Go, running on Kubernetes across major cloud providers (GCP, Azure, AWS). The team also contributes to Prometheus, Grafana Agent, and OpenTelemetry, working with a tech stack primarily in Go, Kubernetes, and cloud object storage.

Responsibilities

• Influence roadmap and drive projects from ideation to operation, iterating on features based on results.
• Design, build, operate, and maintain critical distributed systems, ensuring reliability, performance, and availability.
• Participate in on-call rotations and take ownership of services.
• Contribute to open-source projects and mentor team members.
• Collaborate on design discussions and continuously learn about the cloud product and codebase.

Requirements

• Solid experience with at least one programming language (Go, Python, C, C++, Rust, or similar).
• Proven ability to deliver projects from requirements gathering to product shipment in a self-driven manner.
• Experience with developing cloud-native software or systems engineering.
• Track record of writing clean, robust, and performant software.
• Strong problem-solving skills, customer focus, and pragmatic approach to complex challenges.
• Excellent communication and collaboration skills in a remote environment.

Nice-to-Haves

• Experience with containerization technologies like Kubernetes and Docker.
• Familiarity with Grafana and Prometheus, especially in operational or on-call contexts.
• Exposure to microservices architecture and distributed systems.
• Understanding of on-call responsibilities, SRE tasks, or infrastructure as code.