Data Engineer

We are hiring for Data Engineer:

Overview

Our Data Engineers design, build, and optimize scalable data solutions that support enterprise analytics, operations, and strategic initiatives. These roles focus on modern data engineering practices, including Azure cloud services, Databricks, Spark, Python, SQL, automation, CI/CD, and data governance. They also collaborate across technical and business teams to deliver reliable, secure, and innovative data products. Some assignments may support specialized platforms such as Palantir Foundry or specific business domains (e.g., sourcing, transportation, or supplier data).

Core Responsibilities

Data Engineers across our organization may be responsible for:

• Data Engineering & Solution Development
• Analyze, design, and develop data products, pipelines, and integrations using Azure, Databricks, Spark/PySpark, Python, and SQL
• Build and maintain Delta Lake tables, data models, and ETL/ELT workflows for analytical and operational use
• Optimize pipeline performance (e.g., partitions, caching, shuffles, data skew)
• Implement reusable standards, design patterns, architecture guidelines, and data solution best practices
• Create architectural diagrams, interface specifications, and technical documentation
• Support data ingestion, transformation, prediction, reporting, and alerting needs
• Cloud & Automation (DataOps)
• Provision and manage cloud resources using Terraform (Infrastructure-as-Code)
• Implement CI/CD automation using GitHub and GitHub Actions
• Manage Git-based workflows for notebooks, jobs, and engineering artifacts
• Apply cloud computing skills to deploy fixes, upgrades, and enhancements.
• Implement automation frameworks and tools (e.g., CI/CD, Jenkins, Ansible, Puppet)
• Data Architecture, Strategy & Governance
• Contribute to data strategy, standards, and roadmaps across the enterprise
• Promote reuse of data assets and support cataloging, lineage, and governance tools (e.g., Purview, Unity Catalog)
• Drive digital innovation across SQL, NoSQL, and cloud-based data platforms
• Define migration plans from current to future-state architectures.
• Evaluate and recommend new technologies and approaches
• Cross-Team Collaboration & Support
• Work closely with engineering, product, and business teams, including collaboration with 84.51° where applicable
• Troubleshoot and resolve production issues, including Databricks cluster and job failures
• Deliver quality customer service and support end-user needs
• Mentor team members in data engineering principles and best practices
• (As Needed) Palantir Support

Some roles may include:

• Acting as a Palantir Foundry SME during proof-of-concept efforts
• Advising on data ingestion, ontology modeling, governance, and platform integration
• Aligning Palantir work with Azure, Databricks, and enterprise data strategies
• Providing technical assessments, documentation, and knowledge