À propos
We are looking for a Data Engineer to build the future of our data ecosystem. As we move from traditional SQL databases to a modern Databricks Lakehouse, you will be responsible for designing and implementing the pipelines that power our credit union's decision-making ecosystem. You will bridge the gap between our data systems and our business stakeholders to ensure our data is governed, accessible, and actionable.
Key Responsibilities Cloud Data Transformation: Lead the design and implementation of end-to-end data pipelines, transitioning legacy on-premise SQL workloads into a scalable cloud environment. Lakehouse Architecture & Modeling: Execute a Medallion Architecture to transform raw data into curated datasets for analytics and AI use. You will build the foundational models and establish a comprehensive data dictionary to ensure transparency and adoption by business users. Reliable & Observable Pipelines: Deliver robust data. You will implement proactive monitoring and standardized pipeline patterns to ensure maximum stability. Data Quality & Compliance: Architect and manage data governance to enforce data security, PII masking, and regulatory compliance. You will establish automated data quality with automated alerting on failures. Cross-Functional Support: Partner with teams across the organization to translate business requirements into technical models, occasionally assisting with analytics and reports to meet immediate organizational needs. Requirements
Qualifications
Databricks Mastery: Deep, hands-on experience with Delta Lake, PySpark, and Spark SQL. Modern Data Transport: Proficiency in handling various protocols, including REST APIs and secure file transfers. SQL & Python Expert: Proficiency in Python for automation and advanced data manipulation, paired with expertise in optimizing complex SQL queries. DevOps Literacy: Experience with version control (Git) and CI/CD best practices to ensure code quality and reliable deployments. Financial Domain Knowledge (Preferred): Familiarity with credit union or banking data structures (loans, shares, transactions) is a significant plus. The Builder Mindset: You thrive in environments where you are the first to "lay the bricks" and define the standard for those who follow. Quality Obsessed: You believe that data without governance and observability is just noise; you take pride in the accuracy and lineage of your work. AI-Enhanced Workflow: You stay ahead of the curve by actively leveraging AI tools and LLMs to accelerate development, documentation, and testing processes.
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.