Über
Job Summary:
We are seeking an experienced
Senior Data Engineer
with expertise in Scala, Spark, and real-time streaming to design, build, and maintain scalable data pipelines. The role involves working with cloud data services, big data platforms, and streaming technologies to support analytics and machine learning initiatives.
Key Responsibilities: Design, develop, and maintain
ETL/ELT data pipelines
for both batch and real-time ingestion, transformation, and loading using
Spark (PySpark/Scala)
and streaming tools such as
Kafka
or
Flink . Build and optimize
scalable data architectures , including
data lakes, data warehouses (BigQuery) , and streaming platforms. Optimize Spark jobs, SQL queries, and workflows for
performance, efficiency, and cost-effectiveness . Implement
data quality checks, monitoring, and alerting
to ensure accuracy and consistency. Collaborate with cross-functional teams to support
analytics and machine learning initiatives . Mandatory Skills & Qualifications:
Strong proficiency in
Scala, Python, SQL , and
Spark (Spark SQL, DataFrames, Streaming) . Experience with
PySpark, SparkSQL , and
data modeling . Expertise with
GCP data tools : BigQuery, Dataproc, Vertex AI, Pub/Sub, Cloud Functions. Experience with
messaging queues
like
Apache Kafka
or
Pub/Sub . Knowledge of
data warehousing
(Snowflake, Redshift) and
NoSQL databases . Familiarity with
Airflow, Databricks, Docker, and Kubernetes
is a plus. Good-to-Have Skills:
Hands-on experience with additional cloud services, orchestration tools, or big data technologies. Experience Requirements:
Total IT Experience:
Minimum 8 years GCP Experience:
4+ years of recent hands-on experience
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klicken Sie auf „Jetzt Bewerben“, um Ihre Bewerbung direkt auf deren Website einzureichen.