Evaluation Rubric Writer
Paid in USD
Data & AI operations
New opportunity
Talent pool ExpertContractRemote
PhD Benchmark Question Writer
Write original, hard questions in your field that current AI models cannot answer, with fully worked solutions.
At a glance
-
Paid in USDHourly or per task, in USD. Your rate is agreed per project.
-
Remote, any timezoneWork from anywhere with a stable connection.
-
Flexible, part-timeChoose your own hours.
-
Ongoing, project-based Starts when a matching project opens · Rolling applications
-
Contract · ExpertMultiple openings
About the work
As models pass older tests, labs need new questions that only specialists can write and check. Original, unpublished questions are the only reliable measure.
What you will do
- Write graduate-level questions with a single, verifiable answer
- Provide full reasoning and sources for each answer
- Test questions against models and revise ones that are too easy
- Review questions written by other experts
What we are looking for
- PhD, or doctoral candidate, in any academic field
- Ability to write clear, unambiguous questions
- Professional written English
- Honesty about the limits of your own expertise
Nice to have
- Published research or teaching at university level
- Expertise in a field underrepresented in current benchmarks
Hiring process
- Step 1Profile and CV review
- Step 2AI interview (about 20 minutes)
- Step 3Identity and credential check
- Step 4Short paid trial task
Free to apply. Vettedlance never charges candidates, and will never ask you to pay for training, tests or equipment.
Related roles
RLHF Preference Rater (Generalist)
Paid in USD
Data & AI operations
New opportunity
AI Red-Teamer (Generalist)
Paid in USD
Data & AI operations
New opportunity
Multilingual Search Relevance Rater
Paid in USD
Data & AI operations
New opportunity
Speech Transcription QA Reviewer
Paid in USD
Data & AI operations
New opportunity
Dialect Audio Transcriber (Urdu, Punjabi, Saraiki)
Paid in USD
Data & AI operations
New opportunity