AI Agent Evaluator

What you’ll do

• Run real-life scenarios such as bills, bookings, shopping, groceries, travel, returns, emails, and forms
• Approve or stop AI agents at key decision points
• Screen record every task attempt
• Score completion, quality, control issues, satisfaction, and time
• Compare the performance of two AI agents
• Provide clear written feedback on what worked and what didn’t
• Follow detailed testing instructions consistently

What we’re looking for

Qualification: Bachelor’s degree

• U.S.-based
• Comfortable using AI tools in daily life
• Strong attention to detail and instruction-following
• Excellent written communication
• Active personal AI account and smartphone capable of screen recording

Good to know

Openings

50

• Approx. 1.5 hours per task/run
• Work entirely from your phone
• Flexible schedule
• Project kickoff: October 12
• Onboarding available before kickoff