Data Engineer

Data Engineer

$100,000- $140,000

This position is a hybrid opportunity for a Data Engineer and contract-to-hire in nature.

The hiring team is not open to providing sponsorship now, or in the future and is not open to a corp-to-corp arrangement

Responsibilities:

Develop and maintain scalable data pipelines and workflows using Databricks. Process and integrate structured and unstructured data with Delta Live Tables and Streaming Data Processing. Prepare and manage datasets for machine learning workflows, optimizing for training and inference. Collaborate with data scientists and AI teams to support AI/BI Genie integrations. Implement Snowflake and Star Schema architectures to enhance data storage and query performance. Ensure BI reporting pipelines are compatible with advanced analytics tools. Document data processes and architectures thoroughly. Optimize Spark jobs and workflows for performance and cost-efficiency. Troubleshoot and resolve issues related to data quality and pipelines. Stay current with advancements in data engineering, machine learning, and AI technologies. Requirements:

Databricks Certification (e.g., Databricks Certified Professional Data Engineer) preferred. 3-7 years of experience in data engineering roles. Proficiency in Python, SQL, and Apache Spark. Experience with Delta Live Tables and big data processing frameworks. Understanding of machine learning models, pipelines, and AI tools. Familiarity with AI/BI Genie or similar business intelligence tools. Experience with Data Warehouse Architecture (e.g., Snowflake, Star Schema, Kimball methodology). Knowledge of cloud platforms like AWS, Azure, or GCP. Strong problem-solving skills and ability to work collaboratively.