Team Lead Data Engineer Databricks

IrishJobs

Dublin, Dublin, Ireland

Dublin, Dublin, Ireland

Jetzt Bewerben

Über

Role: Team Lead - Data Engineering (Cloud & Databricks) Location: Dublin Sector: Global Financial Services / Alternative Investments Technical Stack: Databricks, AWS, Spark, Python, Terraform The Opportunity We are seeking a high-caliber Data Engineering Team Lead to spearhead the development and evolution of our enterprise-scale Cloud Data Lakehouse. You will lead a high-performing technical team in building a centralized, high-performance data ecosystem that powers global operations for a premier financial services firm. This is a "player-coach" role requiring deep technical mastery of Databricks internals, Spark optimization, and cloud infrastructure. Core Responsibilities Technical Leadership: Mentor a distributed team of engineers, driving excellence through rigorous code reviews, architectural design sessions, and continuous knowledge sharing. Architecture & Design: Design and implement scalable Lakehouse solutions for complex enterprise data processing and advanced analytics. Pipeline Engineering: Build and optimize high-throughput ETL/ELT workflows using Delta Live Tables (DLT) and Structured Streaming for real-time data needs. Performance Tuning: Deep-dive into Spark job configurations and cluster optimization to ensure maximum efficiency and cost-effectiveness. Governance & Security: Implement robust data governance frameworks using Unity Catalog and manage secure access through cloud-native IAM best practices. Modern DevOps: Manage infrastructure as code (IaC) using Terraform and maintain CI/CD integrity via Git-based workflows. Agile Delivery: Champion Scrum methodologies to ensure timely delivery of complex data products. Required Qualifications Experience: 8+ years in Data Engineering, with at least 3+ years of specialized experience on the Databricks platform. Leadership: Proven track record of managing technical teams and leading complex projects from conception to production. Programming: Expert-level proficiency in Python and Spark. Cloud Ecosystem: Strong hands-on experience with AWS (specifically S3, Lambda, and Glue) or equivalent cloud providers. Data Modeling: Advanced SQL skills and a deep understanding of relational and non-relational data modeling. Frameworks: Mastery of Delta Lake architecture and modern ETL/ELT design patterns. Innovative Mindset: Practical experience using AI-assisted development tools to enhance team productivity and code quality. Preferred Skills (Nice-to-Have) Experience within the Financial Services or Fintech sectors. Knowledge of AI/ML deployment patterns (MLOps). Experience building and consuming RESTful APIs for data distribution. Background in real-time data streaming and event-driven architectures. Why Join This Team? You will be at the forefront of data innovation in a culture that prioritizes professional growth and technical excellence. We offer a flexible working environment, comprehensive benefits, and the opportunity to work on one of the most sophisticated data platforms in the industry.

TLNT1_IJ

Dublin, Dublin, Ireland

Sprachkenntnisse

English

Hinweis für Nutzer

Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.

Jetzt Bewerben