Prompt Engineer: LLM Migration & OptimizationWelo Data • Saint Paul, Illinois, United States
Prompt Engineer: LLM Migration & Optimization
Welo Data
- Saint Paul, Illinois, United States
- Saint Paul, Illinois, United States
About
Remote
Part-Time
Are you an expert at navigating the complex logic of Large Language Models?
Welo Data
is seeking a technical
Prompt Engineer
to lead the end-to-end migration of template workflows into high-performance LLM autoraters.
This isn't just about writing prompts—it’s about engineering a technical bridge. You will use advanced APG/APO tools and manual refinement to ensure our automated systems meet (and exceed) human accuracy baselines.
The Mission: Architecting the Future of Rating
Technical Migration: Take ownership of the workflow for transitioning templates to LLM autoraters.
Optimization Leadership: Run and supervise Automated Prompt Optimization (APO) tools, identifying where logic plateaus and providing the manual "spark" to overcome deadlocks.
Metrics-Driven Accuracy: Continuously measure quality against gold data, calculating critical performance metrics like precision, recall, and $F_1$ scores.
Edge-Case Engineering: Solve complex scenarios by designing manual prompts that handle anti-patterns and broken logic in legacy architectures.
Project Details
Schedule: Part-Time (Flexible hours within project milestones)
Location: 100% Remote (Must be based in the United States)
Employment Type: Freelance / Independent Contractor
Who We Are Looking For
Linguistic & AI Mastery: 2+ years of experience as a Prompt Engineer. You must be comfortable tuning LLMs for structured outputs and complex classification tasks.
Academic Background: BS, MS, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field.
Technical Agility: Fast learner capable of mastering proprietary internal tools and interfaces (like the Goose API) with minimal supervision.
Data Fluency: Strong ability to identify error patterns and use SQL or data analytics tools to analyze model performance.
Preferred Skills
Familiarity with shadowbot disagreement tracking between humans and LLMs.
Hands-on experience with Chain-of-Thought (CoT) and few-shot learning.
Proven ability to draft high-level Launch Certification Documentation.
#J-18808-Ljbffr
Languages
- English
Notice for Users
This job comes from a TieTalent partner platform. Click "Apply Now" to submit your application directly on their site.