Senior Data Engineer London ·

Edgefolio Group

+3
+4
London, England, United Kingdom

+3
+4
London, England, United Kingdom

Find similar jobs

About

We're seeking a Senior Data Engineer to transform our analytics infrastructure from monolithic SQL/PostgreSQL systems into a modern Azure-based data platform. This permanent position focuses on building architecture for long-term evolution, not just migration.You'll bring 5+ years of data engineering experience with proven expertise implementing enterprise-scale data platforms. You'll architect solutions using modern patterns, establish data pipelines and CI/CD practices, and mentor our team while building systems that evolve with emerging technologies.We need someone who builds maintainable, scalable, adaptable systems. We're committed to Azure but not wedded to specific products – we value pragmatic solutions based on actual needs.Structure & ExpectationsReports to: Product organisation – you'll work closely with product teams to align data capabilities with business objectivesLocation: London office 3 days per week, remote work rest of the timeTeam: You'll be the senior technical lead and engineer for data platform initiatives, working with business analysts, engineers, and product stakeholdersTimeline: Initial focus on migration (6 months), then platform evolution and capability expansionCore Technical RequirementsAzure Data Platform ArchitectureMicrosoft Azure data stack expertise (including self-hosting non-Microsoft solutions)Experience with PostgreSQL and SQL Server, including data replication/integration patterns between them (our current architecture)Dimensional and relational data modeling expertiseData Engineering & PipelinesRequired languages: Python and SQL for ELT/ETL pipelinesBatch, streaming, and micro-batching architecturesCDC patterns and incremental load strategies at scaleExperience building ingestion pipelines from diverse external APIs (CRM, analytics, finance, observability tools) to consolidate all organisational data into a unified lakehouseInfrastructure & NetworkingAzure networking and security best practices for enterprise data platformsExperience with event-driven architectures and API integration patternsGovernance, Compliance & SecurityGood understanding and respect of information security requirementsExperience with Microsoft Purview or similar governance frameworksGDPR/data protection implementation, including right-to-be-forgottenRBAC, encryption strategies, audit loggingData lineage and quality enforcementDevOps & QualityTest automation for data pipelines (unit, integration, regression)Monitoring of pipeline runs and errorsPerformance tuning and cost optimisationExperience with git source controlNon-Technical Requirements:Communication & DocumentationGood technical writing – provide examplesStrong presentation skills for technical and non-technical audiencesArchitecture diagrams, data flows, runbooksWhat You'll DoArchitect a modern data platform for long-term business evolutionEstablish engineering best practices for data, governance, and operationsBuild robust batch and real-time processing pipelines (mostly combining a web application’s database with various user analytics data sources and other APIs)Create comprehensive documentation and training materialsTranslate business requirements into maintainable technical solutionsWhy This MattersYou'll establish the foundation for our data platform's next decade. This new platform will be the engine for our next generation of internal and customer products and will empower our business with real-time insights to make faster, smarter decisions. #J-18808-Ljbffr

Nice-to-have skills

PostgreSQL
Python
SQL
SQL Server

London, England, United Kingdom

Work experience

Backend
Data Engineer
DevOps

Languages

English

Notice for Users

This job was posted by one of our partners. You can view the original job source here.

Find similar jobs