Sr Data Engineer
Valid8 Financial, Inc.
- New York, New York, United States
- New York, New York, United States
Über
You’ll work on high-volume, revenue-critical media data while helping modernize how data is stored, processed, and served across the organization. This is a hands-on role for engineers who enjoy building, migrating, and improving platforms, not just maintaining them.
What You’ll Do
Design, build, and maintain scalable, reliable data pipelines that support sell-side media monetization.
Ingest, integrate, transform, and model high-volume media and campaign data from multiple sources, delivering analytics-ready datasets that meet quality, accessibility, and business requirements.
Develop lakehouse-style data models that balance flexibility, performance, and cost efficiency, enabling reporting, analytics, operational workflows, and downstream consumption.
Build, optimize, and maintain ETL/ELT processes for both batch and near-real-time workloads, orchestrated with modern workflow tools (e.g., Airflow, Dagster).
Collaborate with machine learning, client-facing, product, application, and analytics teams to understand requirements, support ML feature productionization, and enable self-service insights.
Ensure data quality, lineage, observability, governance, and security, safeguarding cloud data assets and maintaining trust in revenue-critical systems.
Contribute to platform evolution and migration efforts, including evaluating tools, improving workflows, and reducing architectural complexity.
Continuously optimize pipelines, queries, and storage for performance, scalability, and cost efficiency.
Mentor junior engineers and participate in technical design and architecture reviews.
Detect, investigate, and resolve data anomalies to maintain pipeline reliability and data trustworthiness.
Required Experience & Skills Core Requirements
5+ years of experience in building highly scalable and reliable data engineering and analytics platforms
Strong experience building and optimizing modern data pipelines, architectures, and datasets using Data Lake, Data Warehouse, and Lakehouse paradigms
Advanced SQL, Python, and PySpark skills and experience building analytics-ready data models
Experience using Iceberg, Delta, and Parquet data formats
Experience using Dagster and Airflow orchestration tools
2+ years hands-on experience with cloud platforms, such as GCP & AWS
Experience designing and building data pipelines on object storage-based platforms (e.g., S3, Wasabi, Dremio, Snowflake, Delta Lake) to process and manage large-scale analytical datasets
Solid understanding of data quality, testing, monitoring, and operational reliability
Strong communication skills and ability to work closely with technical and non-technical stakeholders
Experience collaborating with Software Engineers using Agile methodologies to build web applications that access, visualize, and sometimes update big data stores in a hybrid OLAP and OLTP environment.
Bachelor's degree or equivalent work experience (minimum 5 years) in Computer Science or related field
Preferred/Nice-to-Have
Experience in media industry strongly preferred including familiarity with sell-side concepts (linear TV or digital inventory, campaign delivery, pacing, audience-based selling and measurement)
1+ years experience supporting or leading data platform migrations, hybrid architectures, or warehouse-to-lakehouse transitions using Dremio is a strong plus
Experience with Tableau, Metabase, or other data visualization tools
Experience working in an operational environment with time-sensitive customer commitments
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.