S'INSCRIRE

Offres d'emploi

Trouvez des postes près de chez vous, sur site, hybrides ou à distance.
  • Emplois similaires à : Lead Data Engineer Databricks Modernization & Insights
XX
Data Engineer (Ab Initio Databricks Modernization)Diverse Lynx • United States
Postuler Maintenant
XX

Data Engineer (Ab Initio Databricks Modernization)

Diverse Lynx
  • US
    United States
  • US
    United States
Postuler Maintenant

À propos

Job Title: Data Engineer (Ab Initio → Databricks Modernization) - 5 Positions
Location: Wilmington, DE - 5 Days onsite role
Long Term Project
Overview
We are seeking a Data Engineer to lead the modernization of legacy ETL systems by migrating Ab Initio workflows to scalable, modular PySpark pipelines on Databricks. The role involves transforming complex data ecosystems into cloud-native architectures while ensuring data integrity, performance, and reliability.
Key Responsibilities
ETL Modernization & Development • nalyze and migrate legacy ETL workflows from Ab Initio to PySpark-based pipelines • Design and develop scalable data pipelines on Databricks • Refactor monolithic processes into modular, reusable components • Leverage existing enterprise datasets to avoid redundancy
Data Integration & Processing • Build and maintain ETL/ELT pipelines integrating data from Snowflake and other sources • Process and publish enriched datasets for downstream applications • Support batch and near real-time data processing
Data Lineage & Optimization • Create end-to-end data lineage and data flow diagrams • Identify redundancies and drive process consolidation and optimization • Ensure adherence to data governance and quality standards
Testing & Validation • Develop unit, integration, and reconciliation frameworks • Perform dual-run comparisons with legacy systems • Validate outputs in UAT and pre-production environments
Deployment & Operations • Support cutover and migration strategy from legacy systems • Decommission legacy workflows and optimize scheduling (e.g., Control-M) • Develop runbooks, monitoring, and operational documentation
Collaboration • Work with data architects, analysts, and downstream application teams • Coordinate user acceptance testing (UAT/FAT) and stakeholder sign-offs
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
  • United States

Compétences linguistiques

  • English
Avis aux utilisateurs

Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.