Senior Software Engineer for Agentic Coding Evaluation

At a glance

• Rate agreed per projectPaid weekly in USD
• Hourly or per-task contractIndependent contractor · Senior review
• RemoteWork from anywhere with a stable connection.
• Flexible, part-timeChoose your own hours.
• Ongoing, project-based Starts when a matching project opens · Rolling applications

Role overview

Design and grade multi-step coding tasks that test AI agents on real repositories.

Key responsibilities

• Build realistic tasks with tests in real codebases
• Review agent trajectories and mark where they go wrong
• Write rubrics for code quality, security and maintainability

Ideal qualifications

• Five or more years of professional software engineering
• Strong command of at least one of Python, TypeScript, Go, Rust, Java or C++

Nice to have

• Open-source maintainer experience
• Experience with CI and test infrastructure

Compensation and contract terms

• Hourly or per task, in USD. Your rate is agreed per project.
• Hourly or per-task contract. You are engaged as an independent contractor.
• Remote. Flexible, part-time, on your own schedule.
• Paid weekly in USD for approved work.
• Projects can be extended, shortened or ended early depending on project needs and quality.

Application process

1. ApplyCreate a free profile and upload your CV.
2. InterviewAbout 20 minutes. You need a laptop or desktop with a camera and microphone.
3. ID and credential checkWe verify your identity and the qualifications the role needs.
4. Matched to projectsOnce verified, you are invited to paid projects that fit your skills.

Free to apply. Vettedlance never charges candidates, and will never ask you to pay for training, tests or equipment.