Cette offre d'emploi n'est plus disponible
Data Engineer
remoterocketship
- Remote, Oregon, United States
- Remote, Oregon, United States
À propos
Build and maintain data ingestion pipelines using Azure Data Factory (ADF) to move data into the corporate Azure Data Lake and related platforms. Develop and enhance data transformation logic in Databricks notebooks, primarily leveraging SQL-based transformations in existing SQL scripts and stored procedures. Support a medallion data architecture (Bronze, Silver, Gold) with Silver and Gold datasets stored primarily in Parquet format. Assist in migrating SQL Server–based processes (scripts, stored procedures, SSIS logic) running on Azure VMs into Azure-native pipelines and Databricks transformations. Continuously update and enhance existing database solutions and systems based on new and evolving requirements from marketing campaign activation and analytics teams, ensuring data remains accurate, timely, and fit for purpose. Manage multiple concurrent initiatives, including modernization efforts and ongoing enhancement requests, while maintaining platform stability and data quality. Configure Azure Data Factory pipelines to orchestrate Databricks notebooks and Fabric-related processes as applicable. Use Azure DevOps for source control, versioning, and deployment of Azure Data Factory, Databricks, and Fabric assets. Test, monitor, and troubleshoot pipelines to ensure data reliability, performance, and quality. Collaborate with senior engineers, project managers, campaign services, analytics, and platform teams to deliver ongoing data solutions. Participate in the adoption and use of Microsoft Fabric, understanding how Fabric components (e.g., OneLake, Lakehouse, Warehousing) integrate with existing Azure and Databricks workflows. Maintain technical documentation for data pipelines, transformations, and datasets Requirements:
Hands-on experience building or supporting pipelines in Azure Data Factory. Experience developing data transformations in Databricks, preferably using SQL within notebooks. Experience working with Azure Data Lake Storage or similar cloud storage platforms. Familiarity with modern data lake and analytics concepts, including medallion architecture. Experience with SQL Server, including writing, maintaining, and understanding complex SQL scripts and stored procedures. Experience using Git / Azure DevOps for source control. Experience with Python for use in Databricks notebooks and supporting data pipeline operations. Ability to follow established architectural patterns and implementation standards. Strong analytical, troubleshooting, and communication skills. Benefits:
Flexible PTO & Company Holidays Flexible Schedule Continuous Professional Training and Development Mix Of Collaborative & Independent Work Community Outreach Opportunities via Anteriad Cares Professional Global Mentoring Program - Career Guidance From Leadership Great Benefits For You And Your Family Comprehensive medical (choice of 3 plans), dental and vision coverage Company paid short-term disability, long term disability and life Insurance Optional supplemental life, accident and critical illness insurance plans 401K with company match Flexible PTO and generous holiday schedule Fully paid primary caregiver leave (12 weeks) & parental bonding leave (2 weeks)
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre a été publiée par l’un de nos partenaires. Vous pouvez consulter l’offre originale ici.