Über
The Senior Data Engineer will be responsible for designing, developing, and optimizing large-scale data pipelines and analytics solutions within the
Microsoft Azure
ecosystem. The role focuses on building robust ETL/ELT frameworks using
Azure Databricks , PySpark, and Delta Lake while ensuring scalability, performance, security, and data governance best practices.
Role Description / Essential Skills
Data Engineering Development Design, develop, and optimize
ETL/ELT pipelines
using
Databricks (Spark)
and
PySpark . Build and maintain
large-scale distributed data processing systems . Process
structured, semi-structured, and unstructured data
using Spark DataFrames and Delta Lake. Implement
Delta Lake architecture
supporting ACID transactions, time travel, and scalable storage. Optimize pipeline performance and reliability for enterprise-scale data workloads. Azure Cloud Ecosystem
Develop and integrate solutions using:
Azure Data Lake Storage (ADLS) Azure Synapse Analytics Azure Data Factory (ADF) Azure Key Vault Key responsibilities include:
Building
automated workflows
using ADF pipelines and Databricks jobs. Managing and optimizing
Databricks clusters and workspaces
for performance and cost efficiency. Ensuring secure access management and secrets handling using Azure Key Vault. Data Modeling & Architecture
Design and implement
analytical data models , including
star and snowflake schemas . Develop scalable and efficient database structures for reporting and analytics. Enforce
data governance, data quality, and security standards . Collaborate with architecture teams to define
secure, scalable cloud data platforms . Collaboration & Agile Delivery
Partner with
data scientists, business analysts, and product teams
to deliver trusted datasets. Translate business requirements into
detailed technical designs and specifications . Participate in
code reviews , mentor junior engineers, and promote best practices. Work within
Agile/Scrum
delivery frameworks. Required Skills
Azure Databricks (Spark) PySpark Azure Data Factory (ADF) Azure Data Lake Storage (ADLS) Delta Lake Data Modeling (Star / Snowflake schemas) Distributed Data Processing Cloud Data Architecture Desirable Skills
Azure Synapse Analytics Cost optimization and performance tuning Data governance and security implementation
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.