Data Engineer – ML Training InfrastructureSpaitial Ltd. • London, England, United Kingdom
Dieses Stellenangebot ist nicht mehr verfügbar
Data Engineer – ML Training Infrastructure
Spaitial Ltd.
- London, England, United Kingdom
- London, England, United Kingdom
Über
We’re seeking a
Data Engineer
to build the pipelines and infrastructure that fuel our large-scale model training. As the first engineer focused on data, you’ll shape the backbone of how we handle terabytes of multimodal training data (images, video, and 3D). This role is ideal for someone who thrives at the intersection of data systems and machine learning—designing reliable, scalable, and efficient ways to get high-quality data into cutting‑edge training runs.
Responsibilities
Architect and
manage data infrastructure
for large-scale ML training datasets (e.g., Apache, Iceberg, Parquet, Spark).
Build and operate ingestion pipelines
for multimodal data (e.g., images, videos, 3D).
Design data loaders , caching, and serving strategies optimized for ML training.
Partner closely with ML researchers to ensure infrastructure scales with training demands.
Uphold code quality and best practices in testing, CI/CD, and reproducibility.
Key Qualifications
3+ years professional software/data engineering experience with production systems.
Proven experience in
large-scale data processing for ML training
(not just analytics/BI).
Hands‑on with
distributed data
frameworks (e.g., Spark, Beam, Cloud SQL, AirFlow) and modern data formats (Parquet, Iceberg).
Proficiency in cloud platforms (AWS, GCP, or Azure).
Strong Python development skills, including testing and code quality.
Experience building and maintaining CI/CD pipelines.
Preferred Qualifications
Familiarity with ML frameworks (e.g., PyTorch, TensorFlow).
Experience preparing multimodal datasets (images, video, 3D) for ML pipelines.
At SpAItial, we are committed to creating a diverse and inclusive workplace. We welcome applications from people of all backgrounds, experiences, and perspectives. We are an equal opportunity employer and ensure all candidates are treated fairly throughout the recruitment process.
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.