Principal AI Engineer - Context - Agents and Context

Key details

• Own the production improvement loop for Context Engine, understanding how extraction automations, retrieval tools and memory behave based on offline evaluations, customer conversations and telemetry
• Define how the team iterates on agents and skills safely, including versioning and rollout of prompts, skills and automations, regression coverage, staged and shadow evaluation, and guardrails
• Design telemetry needed for data-informed engineering decisions, including what to capture from agent traces, tool calls and knowledge retrieval, how it lands in Elasticsearch, and how it feeds evaluation, dashboards and the feedback loop
• Partner with the data science team on evaluation strategy, including golden datasets and evaluators to gate on quality, latency and cost
• Raise the technical bar across the team by reviewing designs and PRs, mentoring engineers in eval-driven development, and writing technical proposals that shape the roadmap
• Company mission
• Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale, unleashing the potential of businesses and people