Dieses Stellenangebot ist nicht mehr verfügbar
Senior azure data engineer
Adastra Corp
- New York, New York, United States
- New York, New York, United States
Über
Responsibilities
Design, develop, and maintain robust data pipelines using PySpark and SQL on Azure environments (Fabric, Synapse, Databricks or equivalent).
Implement and operationalize Lakehouse architectures (Delta Lake/Parquet, zone-based storage) to support analytics and ML workloads.
Collaborate with data scientists, analysts, and product teams to translate requirements into scalable data models and ETL/ELT solutions.
Optimize query performance, data partitioning, and storage formats for cost and performance on Azure.
Implement data quality, validation, lineage and observability practices across pipelines.
Build CI/CD processes for data solutions, including automated testing, deployment, and rollback for pipelines and infrastructure.
Support and troubleshoot production data issues, perform root‑cause analysis and implement corrective actions.
Contribute to architecture reviews, best practices, and documentation for data platform adoption and governance.
Requirements
6+ years of experience in data engineering or related roles, with a track record delivering production data pipelines.
Strong proficiency in PySpark and SQL for large‑scale batch and streaming processing.
Hands‑on experience with Microsoft Azure data services Azure Fabric, Synapse Analytics, Data Factory, Databricks, ADLS Gen2.
Solid understanding of Lakehouse concepts (Delta Lake, ACID transactions on lake storage, partitioning, compaction).
Experience with data modeling, schema design, partitioning strategies and performance tuning for analytics workloads.
Familiarity with CI/CD for data (Azure DevOps/GitHub Actions), infrastructure as code, and automated testing for pipelines.
Good coding practices including modular design, unit tests, code reviews, and version control.
Strong problem‑solving skills and ability to work independently in a remote, cross‑functional environment.
Excellent communication skills to partner with stakeholders across business and engineering teams.
Nice to Have
Experience with Kafka or other streaming platforms and event‑driven architectures.
Familiarity with dbt, Airflow, or other orchestration and transformation tools.
Experience with Power BI, Synapse serverless SQL pools, or other BI tools for downstream consumption.
Azure certifications (e.g., DP‑203, AZ‑305) or Databricks certification.
Knowledge of ML feature stores, model pipelines or MLOps practices.
Looking for someone with hands on experience with Azure Fabric.
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.