Azure Databricks Engineer/ Architect

Title:
Consultant – Data Engineering (Azure Databricks Data Architect)

Location:
Knoxville, Tennessee, United States (ONLY LOCAL CANDIDATES)

Duration of the Contract:
12+ / Long-term project

Client Industry:
Manufacturing

Skills Required

Core Technical Skills

• Advanced
  Apache Spark
  with strong hands-on expertise in
  PySpark and/or Scala
  , including performance tuning using Spark UI, partitioning strategies, joins, caching, and shuffle optimization
• Deep
  Azure Databricks
  platform expertise, including notebooks, jobs/workflows, cluster configuration, Databricks SQL, and implementation of
  Medallion Architecture (Bronze/Silver/Gold)
• Strong experience with
  Delta Lake
  , including ACID transactions, schema enforcement and evolution, MERGE operations, time travel, and performance-optimized Delta table design
• Unity Catalog and data governance
  experience: catalogs, schemas, tables, permission models, secure access patterns, and data lineage across workspaces
• Advanced
  SQL
  skills for data modeling, complex transformations, aggregations, and BI-ready lakehouse structures
• Solid
  Azure platform
  experience, including:
• Azure Data Lake Storage (ADLS)
• Azure Data Factory or Synapse Pipelines
• Event Hubs / IoT integrations
• Networking, identity and access management
• Azure Key Vault and monitoring
• Data modeling and warehousing
  expertise, including dimensional modeling, star/snowflake schemas, and analytics-focused performance design
• CI/CD and DevOps for data engineering
  , using Git-based workflows with Azure DevOps or GitHub Actions for Databricks pipelines and artifacts

Core Functional & Client-Facing Skills

• Ability to conduct
  client meetings and stakeholder interviews
• Experience creating
  structured questionnaires
  for client discovery sessions
• Strong capability in gathering
  user requirements
  and producing
  data architecture and design documents
  for manufacturing applications on Azure and Databricks
• Basic
  project management skills
  to plan, track, and manage data engineering initiatives
• Excellent
  communication skills
  , with the ability to collaborate effectively with both client teams and internal delivery teams

Job Description

We are seeking a highly experienced
Azure Databricks Data Architect / Consultant – Data Engineering
with 9–12 years of experience to support manufacturing data platforms. The ideal candidate brings deep hands-on Databricks and Azure engineering expertise, combined with strong architecture, governance, and client-facing skills. This role involves designing scalable lakehouse architectures, implementing robust data pipelines, enforcing governance with Unity Catalog, and working closely with stakeholders to translate business requirements into high-quality data solutions.