Arabic (Levantine) AI Evaluation Specialist

Overview

We are seeking  Arabic (Levantine) AI Evaluation Specialists to help assess and improve the performance of advanced AI systems. In this role, you’ll contribute directly to the evaluation and enhancement of large language models (LLMs) by testing how they understand, generate, and respond to Arabic content.

You will craft realistic scenarios, analyze model outputs for quality and safety, and help ensure the technology delivers accurate, culturally appropriate, and reliable results. Your insights will play a key role in shaping smarter AI experiences.

Project Details

Language :  Native fluency in Levantin Arabic

Location :  Remote-Egypt

Project Duration : 3 months

Pay Rate :   $10 USD / Hour

Schedule :  40 hours a week. 8 hours per day Mon-Fri

Start Date :   February 2nd

What You Will Do

• Conduct side-by-side comparisons of AI responses and rate their quality on a 1–5 scale based on established guidelines.
• Design scenario-based and edge-case prompts to evaluate model behavior, including tricky, ambiguous, or incomplete information situations.
• Assess outputs for instruction adherence, factual accuracy, tone, safety, and overall usefulness.
• Develop clear evaluation rubrics and criteria to ensure consistent scoring across tasks.
• Create reliable reference materials (articles, transcripts, reports, etc.) to serve as the source of truth for testing.
• Write well-structured “gold standard” responses that demonstrate the most accurate and helpful answer.
• Identify potential issues such as hallucinations, inconsistencies, or cultural / contextual mismatches.

Qualifications

Nice to Have

Note : Please do not use VPNs or IP-masking tools during the recruitment process — our security system requires accurate regional verification.