Cette offre d'emploi n'est plus disponible
À propos
Position Details
Technical Responsibilities
Design, build, and maintain scalable data pipelines and ETL/ELT workflows using Python and SQL. Architect and implement data ingestion patterns for batch, streaming, and hybrid workloads. Develop and optimize solutions on Databricks, leveraging Spark, Delta Lake, Unity Catalog, DLT, and Workflows. Define and implement medallion (Bronze/Silver/Gold) architecture patterns for transformation and governance. Establish and enforce data quality standards, monitoring, alerting, and observability across the data platform. Collaborate with data scientists, analysts, and business stakeholders to understand requirements and deliver reliable data products. Contribute to and maintain data platform architectural diagrams and extensibility guidance. Leadership Responsibilities
Identify and partner with a Product Owner to establish, curate, and execute a transformation backlog. Co-organize and co-host Sprint ceremonies; lead team solutioning sessions and provide estimates. Lead technical decision-making and prepare demos/walk-throughs for technical and non-technical audiences. Coordinate with DevOps and adjacent teams on code merges, testing, and release management. Document, organize, and lead deployments from development through production; ensure best practices and guidelines are adhered to. Maintain and report scrum metrics and feature forecasts daily; proactively escalate risks and recommendations to the VP of Engineering as appropriate. Interview and onboard new hires; approve and document time off; conduct annual performance reviews and salary adjustment recommendations. Hold regular 1:1s to discover and resolve personnel issues; mentor and coach team members on career growth. Create, own, and educate others on the client sandbox solution architecture; establish and maintain a backlog of changes (application growth, data refreshes, and planned curations). Design data staging frameworks based on real-world scenarios (e.g., month-end reconciliation, homeowner settlement/warranty deed, posting to GL, go-dark practice run-architecture only). Ensure Databricks and client read-only connectivity; develop code that can model extensibility development. Required Qualifications
Technical Skills
Big Data Platforms: Hands-on experience with enterprise data platforms; strong preference for Databricks (Unity Catalog, Delta Lake, DLT, Workflows, Spark optimization). Python: Strong to expert-level proficiency including PySpark, pandas, data validation libraries (e.g., Pydantic, Great Expectations), and production-grade coding practices. Data Ingestion: Expert-level experience with Auto Loader, CDC, streaming ingestion, API integrations, and file-based batch processing. SQL: Advanced skills (complex joins, window functions, CTEs, query optimization) across analytical and OLTP workloads (SparkSQL, T-SQL or similar). Platform Architecture: Understanding of lakehouse architecture, data modeling principles (Kimball, Data Vault), and data governance frameworks. Experience
5+ years in data engineering or related roles, with 2+ years in a formal people-management capacity. Proven track record delivering production-grade data pipelines at scale and leading cross-functional initiatives. Experience with CI/CD for data pipelines and infrastructure-as-code practices. Soft Skills
Strong, proactive daily communication and ownership mindset. Ability to demonstrate technical functionality to technical and non-technical audiences. Clear risk identification, mitigation planning, and follow-through. People leadership with empathy, accountability, and coaching orientation. Preferred Qualifications
Databricks certifications (Data Engineer Associate/Professional). Experience with major cloud platforms (AWS or Azure) and native data services. Familiarity with orchestration tools (Airflow, Prefect, Databricks Workflows). Experience driving data migration or multi-source integration programs. Knowledge of real-time streaming technologies (Kafka, Event Hubs, Kinesis). 2+ years managing engineering teams; formal training or certifications in Agile/Scrum (e.g., CSM, SAFe).
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre a été publiée par l’un de nos partenaires. Vous pouvez consulter l’offre originale ici.