Cette offre d'emploi n'est plus disponible
À propos
Data Engineer
Location: Bethpage, NY
Key Responsibilities: •
Data Pipeline Design and Implementation:
Develop and implement scalable and reliable data pipelines using GCP services like BigQuery, Dataflow, Cloud Storage, and Composer. •
Medallion Architecture Implementation:
Design and implement the bronze, silver, and gold layers within the Medallion architecture to organize and process data effectively. •
Data Storage and Processing:
Implement data storage solutions and optimize data processing workflows using appropriate GCP tools and technologies.
• Data Quality and Security:
Ensure data quality, integrity, and security throughout the data lifecycle, including data validation, cleansing, and access control. •
Collaboration and Communication:
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective data solutions.
• Monitoring and Maintenance:
Monitor and maintain the health of the data infrastructure, troubleshoot and resolve data-related issues, and ensure optimal performance. •
Staying Updated:
Keep up to date with the latest GCP features, data engineering best practices, and advancements in data architecture.
• Data Governance:
Implement data governance strategies to ensure compliance with industry standards and best practices.
Skills and Qualifications: •
GCP Expertise:
Strong knowledge of Google Cloud Platform (GCP) services, including BigQuery, Dataflow, Cloud Storage, and Composer. •
Medallion Architecture:
Solid understanding of the Medallion architecture principles and best practices. •
Data Engineering:
Proven experience in data engineering, including data warehousing, ETL processes, and data modelling. •
Programming Languages:
Proficiency in programming languages like Python, SQL, and potentially Apache Beam (for Dataflow).
• Problem-Solving:
Strong problem-solving and analytical skills to troubleshoot and resolve data-related issues.
• Communication:
Excellent communication and collaboration skills to work effectively with various teams. •
Cloud Architecture:
Knowledge of cloud architecture best practices, including scalability, reliability, and security. •
Data Modelling:
Experience with data modelling techniques and tools.
In essence, a GCP Data Engineer focused on Medallion Architecture is a key player in building and managing data platforms on GCP, leveraging the Medallion pattern to ensure data quality, efficiency, and accessibility for various data consumers.
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre a été publiée par l’un de nos partenaires. Vous pouvez consulter l’offre originale ici.