Über
Senior Data Engineer
Location:
Malvern, PA (Hybrid)
Experience Level:
Level 4 (8+ years)
Role Summary
We are seeking a highly motivated Senior Data Engineer to join the Cost Basis Accounting and Method team. This role is primarily focused on a critical, multi-year
Batch Modernization
effort, moving legacy mainframe batch processes to a modernized AWS cloud-based architecture. The ideal candidate will be an independent contributor and a rockstar developer who is passionate about building scalable data pipelines.
Key Responsibilities
Design, develop, and maintain high-volume data transformation logic primarily using
AWS Glue jobs
written in
Python . Develop custom code and potentially AWS Lambda functions for handling complex logic within the batch processes. Utilize
PySpark
and
SQL
for data querying, filtering, and manipulation against various data stores, including modernized data sources and initial DB2 tables. Collaborate with internal mainframe experts to understand legacy system logic and implement requirements for the modernized batch processes. Engage with the build and deployment pipeline, demonstrating a strong understanding of
DevOps concepts
and proficiency with
Git/GitHub . Handle data ingestion from multiple sources, including various vendors, flat files, CSVs, and APIs. Work closely with a dedicated Tech Lead, but be prepared to operate with a high degree of independence. Required Skills and Qualifications
AWS database experience, e.g. Aurora, Redshift. Proven experience as a Data Engineer with a strong focus on data pipelines and ETL/ELT processes. Expertise in
Python
(estimated 80-90% of development work). Experience with
AWS services , particularly Glue and Lambda. Proficiency in
PySpark
and
SQL
for data handling and querying. Familiarity with
DevOps practices
and the Git/GitHub development workflow. Some experience with
Java
batch processes (estimated 10-20% of development work) is a plus. Experience in dealing with varied data formats and sources (vendors, files, APIs). Prior experience with or understanding of mainframe concepts is a
good-to-have
but not a requirement.
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.