Über
Key Responsibilities
Data Ingestion and Integration:
Develop and maintain data ingestion pipelines using tools like
Azure Data Factory ,
Databricks , and
Azure Event Hubs .
Integrate data from various sources, including APIs, databases, file systems, and streaming data.
ETL/ELT Development:
Design and implement ETL/ELT workflows to transform and prepare data for analysis and storage in the data lake or data warehouse.
Automate and optimize data processing workflows for performance and scalability.
Data Modeling and Storage:
Design data models for efficient storage and retrieval in
Azure Data Lake Storage
and
Azure Synapse Analytics .
Implement best practices for partitioning, indexing, and versioning in data lakes and warehouses.
Quality Assurance:
Implement data validation, monitoring, and reconciliation processes to ensure data accuracy and consistency.
Troubleshoot and resolve issues in data pipelines to ensure seamless operation.
Collaboration and Documentation:
Work closely with data architects, analysts, and other stakeholders to understand requirements and translate them into technical solutions.
Document processes, workflows, and system configurations for maintenance and onboarding purposes.
Cloud Services and Infrastructure:
Leverage Azure services like
Azure Data Factory ,
Databricks ,
Azure Functions , and
Logic Apps
to create scalable and cost‑effective solutions.
Monitor and optimize Azure resources for performance and cost management.
Security and Governance:
Ensure data pipelines comply with organizational security and governance policies.
Implement security protocols using Azure IAM, encryption, and Azure Key Vault.
Continuous Improvement:
Monitor existing pipelines and suggest improvements for better efficiency, reliability, and scalability.
Stay updated on emerging technologies and recommend enhancements to the data platform.
Skills
Strong experience with
Azure Data Factory ,
Databricks , and
Azure Synapse Analytics .
Proficiency in
Python ,
SQL , and
Spark .
Hands‑on experience with ETL/ELT processes and frameworks.
Knowledge of data modeling, data warehousing, and data lake architectures.
Familiarity with REST APIs, streaming data (Kafka, Event Hubs), and batch processing.
Good To Have
Experience with tools like
Azure Purview ,
Delta Lake , or similar governance frameworks.
Understanding of CI/CD pipelines and DevOps tools like
Azure DevOps
or
Terraform .
Familiarity with data visualization tools like
Power BI .
Competency
Analytical Thinking
Clear and effective communication
Time Management
Team Collaboration
Technical Proficiency
Supervising Others
Problem Solving
Risk Management
Organizing & Task Management
Creativity/innovation
Honesty/Integrity
Education
Bachelors degree in Computer Science, Data Science, or a related field.
8+ years of experience in a data engineering or similar role.
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.