XX
Lead Data Engineer - Databricks, PySpark & AWSSGIUnited States
XX

Lead Data Engineer - Databricks, PySpark & AWS

SGI
  • US
    United States
  • US
    United States

À propos

Location:
Houston, TX - The Galleria (100% Onsite) Experience:
8+ years Open to GC holders or US Citizens Overview We are seeking a
Lead Databricks Engineer
to support large-scale data platform initiatives focused on
data modernization, cloud migration, and advanced analytics . This is a hands-on senior role requiring deep Databricks expertise, strong AWS experience, and the ability to partner closely with business and technical stakeholders. The ideal consultant has led complex Databricks implementations end-to-end, can code at a high level, and has experience modernizing legacy data platforms into scalable, cloud-native architectures. Key Responsibilities Lead the design and implementation of enterprise-scale
Databricks
solutions on
AWS Drive
data modernization
initiatives, including
lift-and-shift
and re-architecture of legacy data platforms Build and optimize data pipelines using
Python, PySpark, and SQL Design and manage
Delta Lake
architectures and implement
Unity Catalog
for governance and access control Develop and support
streaming and batch
data workloads Configure and optimize Databricks
clusters and serverless compute
for performance and cost Integrate Databricks with upstream and downstream systems (APIs, data sources, analytics tools) Partner with stakeholders to gather
user and business requirements
and translate them into technical solutions Implement
observability, monitoring, and cost controls , including usage, volume, and pricing metrics Support data domains related to
billing, accounting, pricing, and volume metrics Provide technical leadership, best practices, and mentorship to engineering teams Required Skills & Experience 8+ years of experience in
data engineering , with recent hands-on focus on
Databricks Strong experience deploying Databricks on
AWS Advanced coding skills in
Python, PySpark, and SQL Deep knowledge of
Delta Lake ,
Unity Catalog , and Databricks workspace governance Experience with
streaming data
(Structured Streaming, Kafka, or similar) Strong understanding of Databricks
cluster management, serverless compute, and performance tuning Experience integrating Databricks with enterprise systems and data sources Proven ability to work directly with business and technical stakeholders Experience supporting financial data domains (billing, accounting, pricing, usage metrics) is highly preferred Strong communication skills and ability to lead technical discussions Experience with
AKS / Kubernetes
environments Nice to Have Databricks or AWS certifications Consulting or contracting background in large enterprise environments Work Requirements 100% onsite role in Houston, TX Must be authorized to work in the United States
#J-18808-Ljbffr
  • United States

Compétences linguistiques

  • English
Avis aux utilisateurs

Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.