This job offer is no longer available
About
Location: Washington, DC - Remote position but candidate may have to travel onsite to DC occasionally based on need. Duration: 6 months Need associates with at least 10 years of work exp with a minimum of 5 years of exp as data engineer Skills
MySQL Databricks Pyspark Job Description
Design and develop scalable ETL/ELT pipelines using Databricks and PySpark. Build batch and real-time streaming ingestion frameworks. Develop reusable ingestion and transformation frameworks. Implement Medallion architecture (Bronze, Silver, Gold layers). Develop incremental and CDC-based ingestion pipelines Design and implement real-time streaming pipelines using Kafka and Structured Streaming. Optimize Spark jobs, SQL queries, and streaming pipelines. Implement Delta Lake-based ingestion and transformation frameworks. Tune partitioning, caching, and Spark execution strategies. Strong SQL and data modeling skills. Experience with cloud platforms and distributed systems. Familiarity with CI/CD pipelines and DevOps practices. Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.