AWS Data Engineer Python, DataikuCompunnel • San Francisco, California, United States

This job offer is no longer available

AWS Data Engineer Python, Dataiku

Compunnel

San Francisco, California, United States

San Francisco, California, United States

About

We are looking for an experienced AWS Data Engineer to design, develop, and maintain scalable data pipelines using AWS native technologies. This role involves working closely with technical analysts, stakeholders, data scientists, and other team members to ensure data quality, optimize storage solutions, and support data platform modernization efforts. The ideal candidate will have hands-on experience with cloud migration, data transformation, and automation strategies. Key Responsibilities Lead and support data platform modernization projects. Design and develop robust, scalable data pipelines using AWS native services. Optimize ETL processes for efficient data transformation and performance. Migrate workflows from on-premise environments to AWS cloud, ensuring data quality and consistency. Design automations and integrations to resolve data inconsistencies and quality issues. Perform system testing and validation to ensure successful integration and functionality. Implement security and compliance controls within the cloud environment. Conduct data validation checks pre- and post-migration to ensure completeness, consistency, and accuracy. Collaborate with data architects and developers to document manual workflows and design automation strategies. Required Qualifications Minimum of 7 years of experience in data engineering using AWS technologies (AWS Glue, Python, Snowflake, S3, Redshift). Strong experience in designing and developing scalable data pipelines. Proficiency in Snowflake for data transformations and ETL optimization. Experience with both streaming and batch data pipeline architectures. Familiarity with DataOps concepts and CI/CD pipeline setup on AWS. Hands-on experience with Databricks and a willingness to expand capabilities. Knowledge of Dataiku for data preparation and workflow automation. Strong problem-solving and analytical skills. Bachelor’s or Master’s degree in Computer Science or a related field. Preferred Qualifications Experience with AWS EMR, Lambda, and other cloud-based data engineering tools. Exposure to data governance and compliance frameworks.
#J-18808-Ljbffr

San Francisco, California, United States

Languages

English

Notice for Users

This job was posted by one of our partners. You can view the original job source here.

Find similar jobs