Data Engineer
BGTS LTD
- London, England, United Kingdom
- London, England, United Kingdom
À propos
Data Engineering & Development
* Design, build, and maintain high-quality, scalable, and tested data pipelines.
* Develop and manage Databricks structured streaming pipelines.
* Build and optimize event-driven and real-time data processing solutions.
* Implement and maintain Unity Catalog-based Lakehouse architecture.
* Develop analytics-ready datasets to support business insights and reporting.
Platform & Automation
* Build and manage CI/CD pipelines using Azure DevOps.
* Identify and implement automation opportunities across workflows.
* Ensure reliable and stable data platform operations.
* Apply governance, security, and documentation standards.
Data Quality & Reliability
* Establish the Data Lakehouse as a trusted and reliable source of truth.
* Monitor, troubleshoot, and resolve data incidents.
* Support business users and technical teams with data-related queries.
* Continuously improve platform performance and reliability.
Collaboration & Support
* Work closely with data science, analytics, platform, and business teams.
* Champion data engineering best practices.
* Provide technical guidance and mentorship where required.
* Contribute to a culture of learning, quality, and continuous improvement.
Essential Skills
* Strong experience with Azure Databricks and cloud data platforms.
* Advanced proficiency in Python, PySpark, and SQL.
* Experience developing Spark/Databricks pipelines.
* Hands-on experience with structured streaming and event-driven systems.
* Strong understanding of Lakehouse architecture and best practices.
* Experience with Unity Catalog.
* Expertise in Azure DevOps and CI/CD pipelines.
* Knowledge of data modelling (dimensional/star schemas).
* Experience working in Agile environments.
Desirable Skills
* Exposure to multiple data technology stacks.
* Experience in large-scale enterprise environments.
* Knowledge of security, governance, and compliance frameworks
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.