Senior Data Engineer

Job Title:
Senior Data Engineer – Databricks / PySpark

Job Summary:

We are looking for a skilled
Data Engineer
to design, build, and maintain scalable data pipelines and platforms. The ideal candidate will work closely with data scientists, analysts, and business teams to ensure reliable, high-quality data is available for analytics, reporting, and machine learning.

Key Responsibilities

• Design and implement scalable data processing pipelines using
  Databricks
  and
  PySpark
  .
• Develop and optimize ETL workflows for ingestion, transformation, and storage of large datasets.
• Collaborate with data architects and analysts to ensure data models and transformations align with business goals.
• Troubleshoot performance bottlenecks in data processing and improve runtime efficiency.
• Ensure code quality, data integrity, and adherence to best practices for distributed data systems.

Required Skills

• Strong hands-on experience with
  Databricks
  and
  PySpark
  .
• Proven track record of building production-grade data pipelines in distributed environments.
• Deep understanding of data processing frameworks, job orchestration, and optimization techniques.
• Strong problem-solving skills and the ability to work independently in a fast-paced environment.
• Strong understanding of
  relational and NoSQL databases
• Experience with
  version control (Git)
  and CI/CD pipelines

Soft Skills:

• Strong problem-solving and analytical skills
• Ability to work independently and in cross-functional teams
• Good communication and documentation skills
• Attention to detail and data accuracy

Preferred Qualifications:

• Bachelor's or Master's degree in Computer Science, Engineering, or related field
• Experience with
  Databricks and Spark
• Familiarity with
  real-time data processing
• Experience with
  data governance, lineage, and metadata tools
• Exposure to
  machine learning data preparation