This job offer is no longer available
About
Join our team in building innovative optimizations for home loan data, where we manage millions of data points over the lifetime of loans. We are seeking a skilled professional to design and develop robust ETL/ELT workflows and data pipelines, with a focus on performance and scalability.
Required Skills & Qualifications Applicants must be able to work directly on a W2 basis. Strong understanding of data structures, modeling, and lifecycle management. Hands-on experience designing and managing data pipelines with ETL/ELT expertise. Advanced skills in PySpark for distributed data processing and transformation. Experience with NetApp Iceberg and open table formats for analytics. Knowledge of the Hadoop Ecosystem, including HDFS and Hive. Proficiency in cloud platforms such as GCP (BigQuery, Dataflow) or AWS/Azure. Experience with Delta Lake and Dataplex for governance and metadata management. Strong programming skills in Python, Spark, and SQL. Experience with Apache Airflow for workflow orchestration, including authoring and maintaining DAGs. Strong understanding of relational and distributed database systems. Preferred Skills & Qualifications
Experience with modern database concepts and operational/analytical data stores. Proficiency in optimizing data structures for performance and scalability. Ability to provide thought leadership and share insights with the scrum team. Day-to-Day Responsibilities
Design and develop ETL/ELT workflows and data pipelines for batch and real-time processing. Build and maintain data pipelines for reporting and downstream applications using open-source frameworks and cloud technologies. Collaborate with architects and engineering teams to ensure alignment with target architecture. Apply best practices for data governance, lineage tracking, and metadata management. Develop, schedule, and orchestrate complex workflows using Apache Airflow. Troubleshoot and resolve issues in data pipelines to ensure high availability and reliability. Company Benefits & Culture
Commitment to diversity and inclusion, fostering an environment where all individuals feel valued and respected. Opportunities for professional growth and development. Collaborative and supportive work environment.
For immediate consideration please click APPLY to begin the screening process.
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.