REGISTRIEREN
Zurück zur Stellenangebote
XX
Senior Data EngineerCompunnel • Sunnyvale, California, United States
Jetzt Bewerben
XX

Senior Data Engineer

Compunnel
  • US
    Sunnyvale, California, United States
  • US
    Sunnyvale, California, United States
Jetzt Bewerben

Über

Job Summary Seeking a Data Engineer with Spark & Streaming skills to build real-time, scalable data pipelines using tools like Spark, Kafka, and cloud services (GCP) to ingest, transform, and deliver data for analytics and ML.
Key Responsibilities
Design, develop, and maintain ETL/ELT data pipelines for batch and real-time data ingestion, transformation, and loading using Spark (PySpark/Scala) and streaming technologies (Kafka, Flink).
Build and optimize scalable data architectures, including data lakes, data warehouses (BigQuery), and streaming platforms.
Optimize Spark jobs, SQL queries, and data processing workflows for speed, efficiency, and cost-effectiveness.
Implement data quality checks, monitoring, and alerting systems to ensure data accuracy and consistency.
Required Qualifications
Programming: Strong proficiency in Python, SQL, and potentially Scala/Java.
Big Data: Expertise in Apache Spark (Spark SQL, DataFrames, Streaming).
Streaming: Experience with messaging queues like Apache Kafka, or Pub/Sub.
Cloud: Familiarity with GCP, Azure data services.
Databases: Knowledge of data warehousing (Snowflake, Redshift) and NoSQL databases.
Total IT Experience: Minimum 8 years.
GCP Experience: 4+ years of recent GCP experience.
Preferred Qualifications
Tools: Experience with Airflow, Databricks, Docker, Kubernetes.
#J-18808-Ljbffr
  • Sunnyvale, California, United States

Sprachkenntnisse

  • English
Hinweis für Nutzer

Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.

Jetzt Bewerben