S'INSCRIRE

Offres d'emploi

Trouvez des postes près de chez vous, sur site, hybrides ou à distance.
  • Emplois similaires à : Senior Data Engineer
XX
Senior Data EngineerMarble.com • Lincoln, Nebraska, United States
Postuler Maintenant
XX

Senior Data Engineer

Marble.com
  • US
    Lincoln, Nebraska, United States
  • US
    Lincoln, Nebraska, United States
Postuler Maintenant

À propos

Marble is a technology company founded to revolutionize the food processing industry. Marble is seeking a full-time Senior Data Engineer who is ready for a challenge and eager to design, implement, and support automation solutions that are transforming the industry. As a part of the Marble team, you will leverage cutting‑edge technologies to develop the next generation of automated solutions for food processing, enhancing resilience in the food supply chain.
Job Summary As a Senior Data Engineer at Marble, you will own the design and performance of data pipelines that power everything from real‑time classification dashboards to ML training datasets to operational analytics for production facilities. You will work closely with Software, Infrastructure, and Machine Learning teams to ensure data flows efficiently through our pipelines securely, reliably, and at scale.
You will design for both high‑throughput real‑time ingestion and large‑scale batch processing across on‑prem edge nodes and AWS.
Responsibilities
Architect and build scalable ETL/ELT pipelines for both batch and streaming workloads
Design real‑time ingestion and transformation workflows integrating NATS JetStream and distributed microservices
Develop robust data models and ETL layers for ClickHouse, enabling high‑performance analytics and ML feature extraction
Manage and optimize data storage across AWS S3, ClickHouse, and operational datasets generated on‑prem
Build automation workflows for labeling data, CV pipeline pre‑annotation, dataset generation, and versioning
Ensure data quality, validation, integrity, and lineage, including automated tests and monitoring across pipelines
Collaborate with ML and backend teams to deliver pipelines for training datasets and annotation tools.
Implement scalable compute workloads for large dataset transformations
Define and enforce data governance best practices, including schema evolution, retention policies, and compliance requirements
Monitor and improve data pipeline performance across multi‑region environments
Minimum Qualifications
B.S. or M.S. in Computer Science, Data Engineering, or related field
4+ years of experience building production‑grade data pipelines or distributed systems
Strong proficiency in Python and SQL
Production experience with at least one major distributed compute framework, Apache Spark, Ray, or Apache Airflow (2+ years preferred)
Experience building streaming pipelines or real‑time systems (Kafka, NATS, Redis Streams, or similar)
Deep familiarity with AWS cloud services (S3, Lambda, IAM, EC2, Glue etc.)
Experience with PostgreSQL, MongoDB, ClickHouse or other columnar/NoSQL systems
Strong understanding of data modeling, partitioning, schema evolution, and performance tuning
Understanding of data quality, lineage, orchestration, and governance
Ability to design systems in hybrid environments (on‑prem + cloud)
Excellent communication, documentation, and teamwork skills
Preferred Qualifications
Experience with NATS JetStream, Kafka, or high‑throughput messaging systems
Familiarity with GPU‑based CV pipelines, ML datasets, or annotation workflows
Experience with ClickHouse Materialized Views, Replicated Tables, or S3‑backed storage
Experience working in a regulated, safety‑critical, or high‑uptime environment
Experience with Nomad, Consul, Vault, or HashiCorp ecosystem
Job Type: Full‑time
Location: Lincoln, NE - US, Omaha, NE - US, or Cambridge, MA - US
Team members can expect occasional travel for in‑person meetings and site visits.
Marble is an equal‑opportunity employer. We understand the power of a diverse team, celebrate differences, and promote inclusion.
#J-18808-Ljbffr
  • Lincoln, Nebraska, United States

Compétences linguistiques

  • English
Avis aux utilisateurs

Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.