This job offer is no longer available
GCP Data Engineer
Compunnel
- Lisle, Illinois, United States
- Lisle, Illinois, United States
About
Key Responsibilities
Collect business requirements by working closely with various business, IT, Analyst, and Data Science groups.
Design, develop, deploy, and support high-performance data pipelines, both inbound and outbound.
Model the data platform by applying business logic and building objects in the semantic layer.
Optimize data pipelines for performance, scalability, and reliability.
Implement CI/CD pipelines for continuous deployment and delivery of data products.
Ensure the quality of critical data elements, prepare data quality remediation plans, and collaborate to fix quality issues.
Document the design and support strategy of data pipelines.
Capture, store, and socialize data lineage and operational metadata.
Troubleshoot and resolve data engineering issues.
Develop REST APIs to expose data to other teams.
Mentor and guide junior data engineers.
Required Qualifications
6+ years of experience in data engineering solutions such as data platforms, ingestion, data management, or publication/analytics.
2+ years of experience in Google Cloud Platform (GCP) with services like BigQuery, Composer, GCS, DataStream, or Dataflows.
Expert knowledge of SQL and Python programming.
Experience working with Airflow as a workflow management tool, including building operators to connect, extract, and ingest data.
Experience in tuning queries for performance and scalability.
Experience in real-time data ingestion using GCP Pub/Sub, Kafka, Spark, or similar technologies.
Proven experience working in an agile environment.
Proven experience working in incremental execution through successful launches.
Preferred Qualifications
None specified in the provided text.
Certifications
None specified in the provided text.
#J-18808-Ljbffr
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.