Dieses Stellenangebot ist nicht mehr verfügbar
Über
Location: Irving, TX
Job Description:
We are seeking a skilled
PySpark Data Engineer
to join our team. The ideal candidate will have expertise in big data processing, ETL pipeline development, and cloud-based data engineering solutions. You will work closely with data analysts, data scientists, and software engineers to design, develop, and optimize scalable data processing solutions using
PySpark, Apache Spark, and other big data technologies .
Key Responsibilities: Develop, optimize, and maintain
ETL pipelines
using
PySpark and Apache Spark . Design and implement
big data solutions
for processing large datasets efficiently. Work with structured and unstructured data sources to transform, clean, and enrich data. Optimize Spark jobs for
performance tuning
and
cost efficiency . Collaborate with data scientists and analysts to ensure smooth data integration and availability. Implement
data quality checks, validation, and governance
best practices. Deploy and manage data workflows in
cloud platforms
(AWS, Azure, GCP). Utilize
SQL and NoSQL databases
for data storage and retrieval. Automate data ingestion and transformation processes. Work in an
Agile environment , actively participating in sprint planning and reviews. Troubleshoot and resolve
performance and scalability issues . Required Skills & Qualifications:
Extensive years of experience
in
Big Data Engineering
with a focus on
PySpark and Apache Spark . Strong knowledge of
Python
and distributed computing frameworks. Experience working with
Hadoop, Hive, HDFS, Kafka, and other big data technologies . Proficiency in
SQL, NoSQL, and relational databases (PostgreSQL, MySQL, Cassandra, MongoDB, etc.) . Experience with
cloud platforms
(AWS, Azure, or GCP) and services like
S3, EMR, Databricks, Glue, or Big Query . Knowledge of
data warehousing concepts and ETL development . Hands-on experience with
CI/CD pipelines, Docker, and Kubernetes
is a plus. Strong problem-solving and analytical skills with attention to detail. Excellent communication and teamwork skills.
Muralikrishna
Sarian Solutions, Inc.
Ph: +1 732-790-2266 X 201
murali.k@sariansolutions.com | https://www.sarianinc.com/
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.