Senior AI Engineer - Grafana AI/ML | USA | Remote
Level inferred from job title
Job description
Senior AI Engineer at Grafana Labs building AI-powered observability features, including LLM- and agent-powered workflows for incident detection, triage, and response. The role is remote and open to applicants in Canada time zones.
At a glance
- Employment
- Full-time · Employee (employment agreement)
- Work mode
- Remote · Canada
- Pay
- CAD 129,392 – 217,128 / year
What you'll do
- Take ownership of developing high-performance AI features to help users detect, triage, and resolve incidents using observability data and tools
- Implement a highly iterative process where you quickly prototype, test, and validate with real users, including shipping and evolving LLM- or agent-powered workflows for incident lifecycle management and automated analysis tasks
- Work with data analysts, product managers, and designers to shape AI-driven product features, including integration of agentic components with internal tools, alerting systems, runbooks, and developer workflows
- Use AI and automation tools to enhance both product functionality and your own development workflows
- Communicate effectively and contribute across teams in a highly dynamic and collaborative environment
- Take full ownership of the AI solutions you develop, ensuring they are innovative, scalable, maintainable, and aligned with real user workflows
What we're looking for
- Solid experience building production software systems (backend and / or full stack)
- Self-starter capable of tackling complex engineering problems with minimal supervision
- Familiar with AI technologies and frameworks, focused on delivering high-quality solutions that work in the real world
- Comfortable releasing prototypes, collecting feedback, and iterating with a pragmatic mindset
- Takes ownership and drives projects forward, can deal with ambiguity and define scope where things are loosely defined
- Communicates effectively with peers, product managers, and designers; open to feedback and solutions-oriented
- Experience with LLMs, prompt engineering, and building applications powered by GenAI
- Proven track record of delivering software that made it into production and is actively used by users
- Exposure to working in cloud-native environments (e.g., AWS, GCP, Azure)
- Experience using observability tools to understand and troubleshoot system behavior
Nice to have
- Experience building or working with agent frameworks or multi-agent workflows
- Experience with infrastructure / devops related tooling: Kubernetes, Docker, Terraform or similar for deployments
- Familiarity with model fine-tuning techniques
- Experience building observability tooling
Apply
About Grafana Labs
The company behind Grafana Cloud, the fully managed observability platform
Grafana Labs is the company behind Grafana Cloud, the fully managed observability platform trusted by more than 10,000 organizations to ensure reliability, resolve incidents faster, and optimize telemetry at scale. Built on open source and open standards and designed for interoperability across any stack, Grafana Cloud brings AI to observability and observability to AI. We are a 100% remote company with team members across 40+ countries.
Key facts
- Observability / software
- 100% remote company with team members across 40+ countries
- Grafana Cloud trusted by more than 10,000 organizations
- Backed by leading investors including Lightspeed Venture Partners, Sequoia Capital, GIC, Coatue, J.P. Morgan, CapitalG, and Lead Edge Capital
- Global annual leave policy of 30 days per annum, with 3 days reserved for Grafana Shutdown Days
- In-person onboarding
Partners & customers
What they value
- Transparency
- Autonomy
- Trust
- Collaboration
- Curiosity
- Feedback
- Open source roots
- High trust, low ego