Dieses Stellenangebot ist nicht mehr verfügbar
Über
Must Have Skills: Databricks Spark Hive AWS QuickSight Python Django
Nice to Have Skills:
Detailed Job Description:
Data Engineering & Big Data Development Design and develop scalable, high?performance data pipelines using: 1. Databricks (PySpark/SQL) 2. Apache Spark (batch & streaming) 3. Hive (query optimization, partitioning, bucketing) 4. AWS EMR (PySpark jobs for large-scale data processing) 5. Azure Data Factory (ADF) for ingestion and pipeline orchestration. Build data processing frameworks to handle structured, semi'structured, and unstructured datasets. Develop highly optimized ETL/ELT workflows using Spark, SQL, Python. Create curated data models (Bronze/Silver/Gold) using Databricks Delta Lake. Optimize Spark transformations through: 1. Caching, checkpointing 2. Partition pruning 3. Adaptive query execution (AQE) ? Build DBT models for: 1. SQL-based transformations 2. Automated testing 3. Lineage graphs 4. Data documentation to provide transparency across pipelines.
Minimum Years of Experience: 6 years
Certifications Needed: None
Top 3 responsibilities you would expect the Subcon to shoulder and execute:
Interview Process (Is face to face required?) No. Video call
Any additional information you would like to share about the project specs/nature of work:
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot wurde von einem unserer Partner veröffentlicht. Sie können das Originalangebot einsehen hier.