Dieses Stellenangebot ist nicht mehr verfügbar
Über
Job Description
Location: Alpharetta, Georgia
Experience:8 - 10 Years Mandatory Requirement: 8+ years of hands-on data engineering experience in enterprise environments. Strong expertise in Azure services, especially Azure Databricks, Functions, and Azure Data Factory (preferred). Advanced proficiency in Apache Spark with Python (PySpark). Strong command over SQL, query optimization, and performance tuning. Deep understanding of ETL/ELT methodologies, data pipelines, and scheduling/orchestration. Hands-on experience with Delta Lake (ACID transactions, optimization, schema evolution). Strong experience in data modelling (normalized, dimensional, lakehouse modelling). Experience in both batch processing and real-time/streaming data (Kafka, Event Hub, or similar). Solid understanding of data architecture principles, distributed systems, and cloud-native design patterns. Ability to design end-to-end solutions, evaluate trade-offs, and recommend best-fit architectures. Strong analytical, problem-solving, and communication skills. Ability to collaborate with cross-functional teams and lead technical discussions. Skills: Java,Python,Spark You Are: We are seeking a highly skilled Senior Data Engineer to architect, design, and implement scalable data platforms and pipelines on Azure and Databricks . The ideal candidate will have extensive experience in large-scale data engineering, data modelling, and performance optimization, with a strong focus on building reliable, high-quality data solutions for analytics and operational workloads.
---
Key Responsibilities 1. Data Platform Architecture Engineering
* Architect, design, and implement scalable data platforms and pipelines using Azure and Databricks . * Build and optimize data ingestion, transformation, and processing workflows for both batch and real-time data streams. * Work extensively with Azure Data Lake Storage (ADLS) , Delta Lake , and Apache Spark (Python/PySpark) for large-scale data engineering.
2. ETL/ELT Development Optimization
* Lead the development of complex ETL/ELT pipelines , ensuring high performance, reliability, and maintainability. * Design and implement data models conceptual, logical, and physicalfor both analytical and operational use cases. * Collaborate on data integration and migration projects, working with PostgreSQL, Delta Lake , and other relational or lakehouse systems.
3. Data Governance Quality
* Define and enforce best practices in data governance, quality, and security. * Ensure compliance with enterprise data standards and architecture frameworks. * Troubleshoot production issues, optimize performance, and continuously improve platform efficiency.
4. Collaboration Leadership
* Collaborate with data scientists, analysts, architects, and business teams to translate requirements into scalable data solutions. * Mentor junior engineers, promote coding and design best practices, and contribute to engineering standards and reusable components . * Participate in code reviews, design discussions , and technical documentation .
---
What You Need Experience Expertise
* 8+ years of hands-on data engineering experience in enterprise environments. * Strong expertise in Azure cloud services , including Azure Databricks , Azure Functions , and Azure Data Factory (preferred). * Advanced proficiency in Apache Spark with Python (PySpark) . * Deep understanding of ETL/ELT methodologies , data pipelines , and workflow orchestration . * Proven ability to work with Delta Lake (ACID transactions, schema evolution, optimization). * Solid background in data modelling (normalized, dimensional, and lakehouse models). * Experience with both batch processing and real-time streaming (Kafka, Event Hub, or equivalent). * Strong command of SQL , query optimization, and performance tuning. * In-depth understanding of data architecture principles , distributed systems , and cloud-native design patterns . * Ability to design end-to-end data solutions , evaluate trade-offs, and recommend best-fit architectures. * Strong analytical, problem-solving, and communication skills. * Proven ability to collaborate across cross-functional teams and lead technical discussions.
---
Preferred Skills
* Experience with CI/CD tools such as Azure DevOps and Git . * Familiarity with Infrastructure as Code (IaC) tools Terraform , ARM templates , etc. * Exposure to data governance and cataloging tools such as Azure Purview . * Experience supporting Machine Learning (ML) or Business Intelligence (BI) workloads on Databricks.
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.