Data Architect
Daman
- New York, New York, United States
- New York, New York, United States
À propos
Job Location : Remote
Job Type : Long-term Contract
Role Overview Serve as a full-stack data engineer in the Data Accessibility Program, the foundation platform for the enterprise data portfolio. Design, build, and operate:
AWS-based medallion architecture (Bronze / Silver / Gold)
The data lake and data warehouse
Enterprise ETL/ELT pipelines and data services / APIs
Data governance, cataloging, and data-as-a-service capabilities
The Full-Stack Data Engineer works within the Data Accessibility Program, supporting the enterprise data platform built on
AWS and Apache Iceberg . The engineer designs, builds, and operates the Bronze/Silver/Gold medallion architecture, ingestion frameworks, curated datasets, data services/APIs, and governance‑compliant pipelines.
Core Responsibilities
Design and evolve medallion architecture for the enterprise data platform (Bronze, Silver, Gold).
Define and implement SCD (Type 2) and CDC strategies with data integrity, referential integrity from source systems into S3 and Apache Iceberg.
Perform data mapping from source systems into canonical and analytic models.
Use AWS Transfer Family, AWS Database Migration Service (DMS), AWS Glue, AWS Lambda, EC2, Sonra Flexter, and Aurora PostgreSQL to ingest and land raw data into S3 (Parquet – data partitioning) and Apache Iceberg.
Silver (Curated on Apache Iceberg)
Use Upsolver - Qlik Talend Cloud to build curated Iceberg tables that standardize and conform data across domains.
Apache Iceberg table design and operations: partitioning, schema evolution, compaction, and performance tuning.
AWS & Lakehouse Platform Skills
Amazon S3 for raw, curated, and derived datasets; including partitioning, lifecycle policies, and storage classes.
AWS Glue for ETL jobs, crawlers, workflows, and use of Glue Catalog.
AWS Lambda and EC2 for custom data processing jobs and API hosting.
AWS Lake Formation and AWS Datazone for refined access management and governance.
HashiCorp Terraform to provision and manage AWS resources used by the data platform.
Data Services & APIs
Design and build data services / APIs that expose curated Silver data as reusable data products.
Implement Python-based APIs and services on AWS compute (Lambda) using API Gateway.
Ensure services implement observability (logs, metrics, traces) and align with operational SLAs.
Security, Governance & Operations
Apply tokenization, encryption, masking, and redaction of sensitive data.
Maintain accurate AWS Glue Catalog entries for all technical assets (S3, Iceberg, Aurora).
Monitor and optimize performance and cost across S3, Glue, Aurora, Iceberg, and APIs.
Seniority level
Executive
Employment type
Contract
Job function
Information Technology
Industries
IT Services and IT Consulting
#J-18808-Ljbffr
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.