SE CONNECTERS'INSCRIRE
Retour aux emplois
XX
(Fermé)Zortech Solutions

Lead Databricks Engineer with Pyspark & Python Exp---USA

  • +3
  • +4
  • US
    New York, New York
Trouver des emplois similaires
  • +3
  • +4
  • US
    New York, New York

À propos

Role: Lead Databricks Engineer with Pyspark & Python Exp
Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office)
Job Description:
Inperson interview is must, Expenses will be paid for in-person if not local
(Financial background is must)
Major Responsibilities:
  • Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
  • Design, develop, and deploy Databricks jobs to process and analyze large volumes of data.
  • Collaborate with data engineers and data scientists to understand data requirements and implement appropriate data processing pipelines.
  • Optimize Databricks jobs for performance and scalability to handle big data workloads.
  • Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.
  • Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.
  • Demonstrated proficiency with Data Analytics, Data Insights
  • Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process
  • Azure Synapse/Bigquery/Redshift is good to have.
  • Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.

Skills:
  • 5+ years' - Strong experience in Finance / Banking industry - Capital markets, investment banking - Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
  • 10+ years - Enterprise Data Management
  • 10+ years - SQL Server based development of large datasets
  • 5+ years with Data Warehouse Architecture, hands-on experience with Databricks platform. Extensive experience in PySpark coding.
  • Azure Synapse/BigqExperience with Cloud based data architectures, messaging, and analytics
  • Superior communication skills
  • Cloud certification(s)
  • Any experience with Regulatory Reporting is a Plus

Education:
  • Minimally a BA degree within an engineering and/or computer science discipline
  • Master's degree strongly preferred

Compétences idéales

  • PySpark
  • Python
  • SQL
  • Redshift
  • New York, New York

Expérience professionnelle

  • Data Engineer
  • Data Infrastructure
  • Data Analyst

Compétences linguistiques

  • English
Trouver des emplois similaires