SIGNUP

Job Opportunities

Find jobs near you, whether onsite, hybrid, or remote.
  • Similar Jobs to: AI QA Trainer - LLM Evaluation - Freelance Project
XX
AI QA Trainer - LLM Evaluation - Freelance ProjectInvisible Expert Marketplace • New York, New York, United States
Apply Now
XX

AI QA Trainer - LLM Evaluation - Freelance Project

Invisible Expert Marketplace
  • US
    New York, New York, United States
  • US
    New York, New York, United States
Apply Now

About

AI QA Trainer – LLM Evaluation Are you an AI QA expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into enterprise-grade platforms. With rigorous evaluation data, tomorrow’s AI can democratize world-class education, keep pace with cutting-edge research, and streamline workflows for teams everywhere. That quality begins with you—we need your expertise to harden model reasoning and reliability.
Responsibilities On a typical day you will converse with the model on real-world scenarios and evaluation prompts, verify factual accuracy and logical soundness, design and run test plans and regression suites, build clear rubrics and pass/fail criteria, capture reproducible error traces with root‑cause hypotheses, and suggest improvements to prompt engineering, guardrails, and evaluation metrics (e.g., precision/recall, faithfulness, toxicity, and latency SLOs). You’ll also partner on adversarial red‑teaming, automation (Python/SQL), and dashboarding to track quality deltas over time.
Qualifications A bachelor’s, master’s, or PhD in computer science, data science, computational linguistics, statistics, or a related field is ideal; shipped QA for ML/AI systems, safety/red‑team experience, test automation frameworks (e.g., PyTest), and hands‑on work with LLM eval tooling (e.g., OpenAI Evals, RAG evaluators, W&B) signal fit. Skills that stand out include evaluation rubric design, adversarial testing/red‑teaming, regression testing at scale, bias/fairness auditing, grounding verification, prompt and system‑prompt engineering, test automation (Python/SQL), and high‑signal bug reporting. Clear, metacognitive communication—“showing your work”—is essential.
Pay & Benefits We offer a pay range of $6‑to‑$65 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.
Employment Type Contract
Workplace Remote
Seniority Level Mid‑Senior Level
#J-18808-Ljbffr
  • New York, New York, United States

Languages

  • English
Notice for Users

This job comes from a TieTalent partner platform. Click "Apply Now" to submit your application directly on their site.