Data Engineer

Purpose Of Position
We are looking for an experienced and highly skilled Data Engineer to join our Data Governance Team. The ideal candidate will have a strong background in data engineering with specific experience in Databricks. You will be responsible for designing, building, and maintaining scalable data pipelines and architectures to enable advanced analytics and data-driven decision-making across the organization.

Essential Job Functions

• Design and develop scalable data pipelines using Databricks, ensuring efficient data processing and integration from various sources.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions that meet their needs.
• Implement best practices for data management, storage, and retrieval, focusing on optimizing performance and ensuring data integrity.
• Work with large datasets and perform data cleansing, transformation, and aggregation to support analytics and reporting functions.
• Monitor and troubleshoot data pipeline performance, ensuring high availability and reliability.
• Stay updated with the latest advancements in data engineering and Databricks technologies to continuously improve our data infrastructure.
• Document data processes and workflows, ensuring transparency and knowledge sharing across the team.
• Ensure data security and compliance with regulatory requirements and industry best practices.
• Perform other duties as directed.

Knowledge, Skills And Experience Requirements

• Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent experience
• 3+ years of experience as a Data Engineer, preferred proven expertise in using Databricks.
• Strong proficiency in data processing tools and languages such as SQL, Python, and Spark.
• Experience with cloud platforms, particularly Azure, and integration with Databricks.
• Understanding of data architecture principles and experience with ETL tools and methodologies.
• Excellent problem-solving skills, with the ability to manage multiple priorities in a fast-paced environment.
• Strong communication skills, with the ability to collaborate effectively with cross-functional teams.

Working Conditions
This is a full-time position operating on a hybrid schedule with three days per week in the 275 7th Ave, NY office.