This job offer is no longer available
About
Required Skills: GCP Python Big Query Apache Airflow
Job Description*
Purpose The Software Engineer Lead IT is responsible for leading the design of technology solutions to ensure they meet all resiliency, stability, non-functional, functional and operational readiness use cases. This position is responsible for owning the technology solutions within their area, including proactive monitoring and alerting, technological roadmap and communication of overall health and future sustainability.
Responsibilities: Lead design to ensure the technology solutions meet all resiliency, stability, non-functional, functional and operational readiness use cases? Lead ongoing development, maintenance and support of existing systems and platforms? Manage work direction, lead, influence, and provide feedback to other team members?
Overview Design, build, test & support data integrations for Partner IQ
Experience • 8-10 years in Data Engineering • GCP experience (or Cloud Based Platform such as AWS/Azure) • Data Pipeline Development • Understanding of Data Governance Principles
Skillset • GCP BigQuery: Query optimization, partitioning, clustering, data modeling • Cloud Storage: Bucket management, lifecycle policies, data archival • Cloud Composer: Apache Airflow for workflow orchestration • Python: Primary language for data engineering to build pipelines • SQL: Advanced querying, window functions, CTEs • Bash/Shell: Automation and scripting • Monitoring: Cloud Logging, Cloud Monitoring • Git: Version control • Data Warehousing: Dimensional modeling, ETL/ELT patterns • Data Quality: Validation, testing, monitoring • Performance Optimization: Query tuning, cost optimization
Preferred • Dataflow: Apache Beam, Streaming/Batch Processing Pipelines • Pub/Sub: Event-Driven Architecture, Message Queuing • Cloud Functions: Serverless Data Processing Triggers • Real-time Processing: Streaming Analytics • PBM/Healthcare Experience • Strong Analytical & Problem-Solving Skills • Excellent Communication Skills • Ability to Effectively Collaboration with Cross-Functional Teams
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.