This job offer is no longer available
Machine Learning Engineer for AI Model Benchmarking
SaidGig
- United States
- United States
About
This position offers an exciting opportunity for experienced machine learning engineers and researchers to act as human baseliners for evaluating open-ended machine learning research tasks. Your role will involve performing realistic AI R&D problems, providing essential human reference points to interpret AI agent performance. You will work independently in a sandboxed environment, utilizing your preferred tools and workflows to complete self-contained ML research tasks, which will serve as benchmarks for evaluating frontier-model agents. Key Responsibilities
Attempt open-ended machine learning research tasks under a fixed time and compute budget. Work independently in a sandboxed Linux environment with internet access. Utilize your preferred tooling, including IDEs and AI coding assistants such as Cursor, Claude Code, and ChatGPT. Record your full working session via screen recording. Complete a short pre-task and post-task questionnaire. Submit your final work product, screen recording, and completed questionnaires; successful candidates will be considered for longer commitments. Commitment
Selected candidates are expected to commit to a minimum of 20 hours per week, with more availability strongly preferred. Qualifications
Candidates must meet all of the following requirements: 3+ years of machine learning experience (time spent in a PhD program counts toward this requirement; undergraduate and master’s experience does not). Graduated from a top-100 university or worked at a FAANG or comparable company. Experience with at least one major ML framework such as PyTorch, JAX, or TensorFlow. Deep, hands-on expertise in at least one of the following focus areas: Pretraining under tight data and compute budgets. PPO, reward shaping, custom gym/gymnasium environments, and throughput tuning. Full fine-tuning, LoRA, QLoRA, DPO, RLHF, RLAIF, and distillation. Large-scale corpus filtering, deduplication, subsampling, and benchmark contamination avoidance. Architecture design under strict parameter-count or size constraints. Modifying pretrained architectures, including attention patterns, pooling heads, or training objectives. Contrastive training for embedding or retrieval models. Generative vision or video modeling. Multilingual or low-resource language experience. Image or video data pipelines at scale. Experience balancing competing model objectives such as safety and capability. Prior work as an ML evaluator, red-teamer, or baseliner. Required Domain Expertise
Candidates must have strong practical experience in at least one of the following: Pretraining: training transformer language models from scratch. Reinforcement learning: training agents in custom or existing environments. Post-training: fine-tuning and aligning LLMs. Dataset curation: building and cleaning large text corpora for LLM training. Model architecture: designing and modifying neural network architectures. Logistics (Work Trial Requirements)
One baseline attempt per contractor per task. Each task may only be attempted once by a given contractor. All work is confidential and covered by NDA. Compute and environment are provided; no personal GPU is required.
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.