Back to roles
Patronus AI logo
Patronus AI

Technology, Information and Internet · Posted 6h ago

SRE Expert, AI Evaluation Sessions

Solve live reliability scenarios on a shared screen and shape how frontier AI models are tested

Within a regionAnywhere in United States and European Union, as long as you like

The listing names the US and EU as where you can be based, with a few hours of overlap with their working hours.

How long can you stay where you're going?
  • Paid trial session
  • Extension for strong performers
  • No AI background needed

The role

Patronus AI contracts experienced SREs, platform and observability engineers for about 10 hours a week. You'll work incidents live with researchers, and your solutions become evaluation and training data.

What you'll do

  • 1Work through incident and infrastructure scenarios live with a researcher
  • 2Explain your reasoning, including fixes you considered and rejected
  • 3Review drafted scenarios and flag what's unrealistic or underspecified
  • 4Write acceptance criteria, including tempting wrong moves under pressure
  • 5Write short post-session notes and act on reviewer feedback

What they're looking for

Must-haves

  • 3+ years running production systems, with on-call in the last two years
  • Hands-on with Kubernetes, Prometheus and Grafana (or Datadog, New Relic)
  • Experience with a log stack (Loki, ELK, Splunk), Terraform, Helm and CI/CD
  • Comfortable in Python, Go or Bash and reading unfamiliar code
  • Clear spoken English and reliable availability for scheduled live sessions

Nice to have

  • Experience writing runbooks, post-mortems or SLO definitions
  • Familiarity with how AI coding agents fail on operations work
  • Experience creating incident training material

Skills and platforms

  • Kubernetes
  • Prometheus
  • Grafana
  • Helm
  • Terraform
  • CI/CD
  • Site Reliability Engineering
  • Incident Response
  • Python
  • Go
  • Bash
  • Observability
  • LogQL
  • PromQL
  • Infrastructure as Code
  • Debugging

The company

Patronus AI logo

Patronus AI

Technology, Information and Internet

Patronus AI is a model behavior research lab developing training data and simulated environments for frontier AI. Our work spans evals and digital world models for agents operating in software.

Summarised by Jobs That Travel from Patronus AI's listing.

Similar roles

Browse all roles
Patronus AI logo

Patronus AI

Technology, Information and Internet

6h ago

Payments Processing Engineer, AI Coding Expert

Work live with AI researchers on payments code, from idempotent retries to ledger reconciliation

Within a region

Paid trial session

$150 /hr

Contract / freelance · Remote, United States

A few hours of overlap with US or EU hours

Gramian Consulting logo

Gramian Consulting

IT Services and IT Consulting

5d ago

Energy Regulatory & Utilities Attorney, AI Training

Review and sharpen AI-generated legal analysis on FERC, state PUC and ISO/RTO matters

Anywhere

Pay not listed

Contract / freelance · Remote · Worldwide

Async: no set hours

Gramian Consulting logo

Gramian Consulting

IT Services and IT Consulting

8d ago

Machine Learning Research & Engineering Specialist

Debug, optimize and validate ML training and inference code for an AI training project

Anywhere

Pay not listed

Contract / freelance · Remote · Worldwide

Async: no set hours