This job offer is no longer available
About
Location:
Malvern, PA (Onsite/Hybrid likely)
Duration:
6 months (extendable)
Pay Range: $54-58/Hour on W2 only
Note: Applicant must be willing to work on W2.
Request-ID: 79469-1
Role Overview:
This role is for a senior data engineer who can design and build scalable, real-time data pipelines on AWS using Python and Spark. This is a senior-level data engineering role focused on building scalable, real-time (streaming) data pipelines using Amazon Web Services and Python-based technologies.
Key Responsibilities:
Build and maintain event-driven data pipelines using AWS services such as Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR.
Develop ETL/ELT workflows using Python and PySpark, ensuring performance, scalability, and cost efficiency.
Implement and optimize Spark-based data transformations, partitioning strategies, and data processing frameworks.
Design and manage data lake and warehouse structures using S3, Glue Catalog, Athena, and/or Redshift.
Build streaming solutions with checkpointing, stateful transformations, idempotency, and schema evolution.
Ensure high standards of data quality, observability, monitoring, and alerting (CloudWatch, Datadog, etc.).
Implement data security best practices including IAM, encryption (KMS), networking, and governance.
Create reusable frameworks, internal libraries, and CI/CD pipelines for automated deployments.
Collaborate with data scientists, analysts, and business teams to deliver well-modeled, reliable datasets.
Lead design reviews, mentor junior engineers, and contribute to engineering best practices.
Required Skills:
Overall 8+ yrs of experience
5+ years of professional experience in Data Engineering.
Experience of working on Java is an advantage
Strong expertise in Python and PySpark for large-scale data processing.
Advanced hands-on experience with AWS (S3, Glue, EMR, Lambda, Step Functions, Kinesis/MSK, DynamoDB, Athena, Redshift).
Deep experience building event-driven and streaming data pipelines.
Strong SQL experience for analytical and ETL workloads.
Hands-on experience with workflow orchestration tools such as Airflow or Step Functions.
Experience with CI/CD, Git, and Infrastructure-as-Code (Terraform or CloudFormation).
Strong understanding of distributed systems, Spark performance tuning, data modeling, and cloud cost optimization.
Knowledge of data security, encryption, networking, and compliance best practices in cloud environments.
Company Benefits & Culture:
Competitive salary and benefits package.
Opportunities for professional growth and advancement.
Dynamic and collaborative work environment.
Appreciate your quick response and please feel free to reach me out for any query you may have.
Kind Regards,
Vibhor Babbar
Manager – Staffing
Artech is the #1 Largest Women-Owned IT Staffing Company in the US!
Call/Text: (973) 394-3916
Email:
| LinkedIn:
360 Mt. Kemble Avenue, Suite 2000 Morristown, NJ 07960
Website:
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.