Dieses Stellenangebot ist nicht mehr verfügbar
Über
6+ months Location:
100% Remote
Requirements:
Client wants someone who has billing domain knowledge experience on all three clouds azure/AWS and Google This engineer builds the foundational data models that ingest and normalize raw billing data from all cloud providers into the data platform. 4-6 years of data engineering experience This role requires deep expertise in ETL/ELT patterns, handling schema evolution, and building robust pipelines at scale. The complexity of cloud billing data (billions of rows, late-arriving data, provider schema changes) demands someone who's solved these problems before Deliverables:
Source System Connectors: Build ingestion pipelines for AWS CUR (S3), GCP Billing Export (BigQuery), and Azure Cost Management Exports (Blob Storage) Staging Data Models: Create staging tables that preserve raw billing data with appropriate partitioning and clustering Normalized Base Models: Design and implement a unified schema that normalizes billing data across all three providers Data Quality Framework: Implement data quality tests, freshness checks, and reconciliation models Schema Evolution Handling: Build processes to detect and adapt to provider schema changes Engineering Responsibilities:
Design incremental ingestion logic handling daily/monthly billing updates Implement late-arriving data handling (billing adjustments, credits) Build reconciliation models comparing processed totals to provider invoices Create data contracts and documentation for downstream consumers Optimize model performance for large-scale billing datasets (partitioning, clustering) Implement CI/CD for data model deployments and testing
26-00114
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.