Talent pool Senior reviewContractRemote

Senior Software Engineer for Agentic Coding Evaluation

Design and grade multi-step coding tasks that test AI agents on real repositories.

At a glance

  • Paid in USDHourly or per task, in USD. Your rate is agreed per project.
  • Remote, any timezoneWork from anywhere with a stable connection.
  • Flexible, part-timeChoose your own hours.
  • Ongoing, project-based Starts when a matching project opens · Rolling applications
  • Contract · Senior reviewMultiple openings

What you will do

  • Build realistic tasks with tests in real codebases
  • Review agent trajectories and mark where they go wrong
  • Write rubrics for code quality, security and maintainability

What we are looking for

  • Five or more years of professional software engineering
  • Strong command of at least one of Python, TypeScript, Go, Rust, Java or C++

Nice to have

  • Open-source maintainer experience
  • Experience with CI and test infrastructure

Hiring process

  1. Step 1Profile and CV review
  2. Step 2AI interview (about 20 minutes)
  3. Step 3Identity and credential check
  4. Step 4Short paid trial task

Free to apply. Vettedlance never charges candidates, and will never ask you to pay for training, tests or equipment.

Related roles

Software Engineer for AI Code Evaluation

Paid in USD

Engineering & software

New opportunity

Data Scientist / ML Engineer

Paid in USD

Engineering & software

New opportunity

Civil Engineer

Paid in USD

Engineering & software

New opportunity

Electrical Engineer

Paid in USD

Engineering & software

New opportunity

Mechanical Engineer

Paid in USD

Engineering & software

New opportunity

Geologist / Mining Engineer

Paid in USD

Engineering & software

New opportunity

All engineering & software roles