Dieses Stellenangebot ist nicht mehr verfügbar
AI 101 Capstone Project Reviewer & Learning Evaluator
LaunchCode
- Saint Louis, California, United States
- Saint Louis, California, United States
Über
At a Glance
Type: Part-Time 1099 Contractor — Remote (all U.S. states except CA)
Compensation: $30/submission flat rate
Volume: ~250 submissions estimated, 20–30 min each (est. 80–125 hours total)
Experience: 3–7 years hands‑on automation experience, or equivalent
Schedule: Flexible; work on your own time within review windows
About This Opportunity LaunchCode's AI 101 course teaches professionals to build AI‑powered automations. The course culminates in a capstone project where students build a Zapier‑based automation, make a business case for its value, and present their work as an 8–12 slide PDF deck.
An AI agent scores each submission across ten rubric criteria spanning two categories (Content and Presentation), then generates written feedback. Your job is to validate that output—correcting scores, rewriting weak feedback, and applying the professional judgment the AI can't.
Students apply four analytical frameworks across their projects; onboarding covers each in detail.
The central challenge of this role is anchoring bias: reviewers who lack domain depth tend to accept the AI's score rather than override it, even when it's wrong. We need someone with enough expertise across automation design, business case construction, measurement methodology, and responsible AI to catch what the AI gets wrong—especially at the Proficient/Exemplary boundary, where the difference requires real professional judgment.
Key Responsibilities
Validate AI scores across all ten rubric criteria; adjust where the AI has mis‑scored or missed nuance
Rewrite AI feedback that is generic, unclear, or misses a learning opportunity — feedback must be specific and actionable
Apply particular scrutiny to: comparative analysis scoring (AI over‑credits surface‑level comparisons); measurement data quality (AI accepts weak methodology); and responsible AI scoring (AI awards credit for naming risks without a credible mitigation plan)
Distinguish Picking the Right Problem (task selection quality) from Strategic Automation (step prioritization quality) — these are separate rubric criteria and must be scored independently
Flag submissions with structural gaps the AI may miss: missing architecture slide, absent speaker notes, slide count outside 8–12, or exposed credentials (API keys, tokens) in screenshots
Flag systemic AI scoring patterns to the course team; participate in periodic calibration checks
Required Skills & Qualifications
Zapier (required): Must be able to evaluate multi‑step Zaps and Agents across all five workflow components — trigger, actions, handoffs, data mapping, and output — and distinguish a functional workflow (Proficient) from one that handles edge cases and could be handed to a colleague with minimal setup (Exemplary)
Workflow & automation design: Able to assess architecture diagrams, identify logical gaps, and evaluate whether automation choices target the highest‑impact steps
Business acumen: Able to evaluate ROI cases, including the distinction between capacity savings and direct cost savings; comfortable assessing whether a student's projected annual value is credible
Data literacy: Able to assess whether deployment data (quality, speed, cost) is credible, methodology is sound, and data visualizations actively support the argument rather than just accompany it
Responsible AI awareness: Familiar with data privacy, algorithmic bias, human‑in‑the‑loop design, and graceful failure documentation; able to distinguish a genuine mitigation plan from a performative one
AI output evaluation: Experience QA'ing AI‑generated scores or assessments in a professional context is required; you must be comfortable overriding AI judgment and articulating why
Presentation evaluation: Able to assess narrative arc, visual design quality, and slide‑and‑speaker‑note coherence for a non‑technical business audience
Written communication: Able to write developmental feedback for adult learners that is specific, encouraging, and growth‑oriented
Preferred Skills & Qualifications
Experience teaching, coaching, or providing feedback in a bootcamp, corporate training, or adult learning context
Background in operations, consulting, or a field where you've personally built and defended automation ROI cases
Familiarity with other automation tools (Make, n8n, Power Automate) in addition to Zapier
Experience evaluating presentations or slide decks in a professional review capacity
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.