Lead Data Engineer
Perficient
- Minneapolis, Minnesota, United States
- Minneapolis, Minnesota, United States
About
Senior Data Engineer (open to Principal level)
to lead the modernization and ownership of a critical data pipeline within a large-scale healthcare analytics environment. This role will focus on transitioning legacy SAS-based pipelines to modern
Python/PySpark on Databricks , while driving engineering best practices and scalable data solutions. This is a
hands‑on engineering role
with a strong emphasis on development capabilities. Candidates with application development experience and exposure to AI/automation technologies will stand out. Key Responsibilities
Lead the
modernization of data pipelines
from SAS to Python/PySpark on Databricks Own and evolve a
mission‑critical HEDIS data pipeline
used for performance measurement and reporting Design, build, and optimize
scalable data pipelines
in a distributed environment Collaborate with SMEs during an initial
knowledge transfer period , with eventual full pipeline ownership Develop, schedule, and automate end‑to‑end data workflows Ensure data quality, reliability, and performance across large datasets Partner with cross‑functional teams and analytics vendors to deliver high‑quality data outputs Contribute to
best practices in version control, CI/CD, and agile development workflows Required Qualifications
Hands‑on experience with
Python (scripting and application development) Expertise in building and managing
data pipelines and ETL workflows Proficiency with
Databricks
(notebooks, workflows, cluster management) Solid experience with
AWS services
including S3, Lambda, Glue, and EC2 Strong
SQL skills
for complex transformations and data extraction Experience with
pipeline orchestration and automation Familiarity with
version control systems (Git)
in a collaborative environment Experience managing work via
issues, epics, and agile tooling Preferred / Nice‑to‑Have
Experience with
AI, machine learning, or automation frameworks Background in transitioning
legacy systems to modern data platforms What are we Looking For (Priority Order)
Application development experience, especially
Python scripting Expertise in
AI or automation
(highly desirable bonus)
#J-18808-Ljbffr
Languages
- English
Notice for Users
This job comes from a TieTalent partner platform. Click "Apply Now" to submit your application directly on their site.