About
Job Title:
Lead Data Engineer Department:
Enterprise Technology - Data to Insights (D2I) Position Open To:
All Applicants Weekly Scheduled Hours:
40 FLSA Status:
Exempt Location:
Texas Earliest Start Date:
Immediately Position Duration:
Expected to Continue Until Dec 19, 2026 Summary:
This is a fixed-term position with a 1-year term from the start date and potential for extension. The role supports a flexible work arrangement, including the possibility of 100% remote work with approval for out-of-state applicants. Travel is limited to training events. Employer/Benefits:
The University of Texas at Austin and Enterprise Technology offer a comprehensive benefits package, including health benefits, retirement plans, paid time off, training opportunities, tuition assistance, and other programs. See details at the UT Austin benefits pages referenced in the original posting.
Purpose The Lead Data Engineer for the UT Data Hub improves university outcomes and advances the UT mission by increasing the usability and value of institutional data. You will lead senior data engineers and data engineers to create complex data pipelines within UT''s cloud data ecosystem to support academic and administrative needs. You will help build and run a modern data hub to enable data-driven decision making and will collaborate across teams and external partners.
Responsibilities
Technical Leadership:
Design, architect, and deliver production-grade, scalable data pipelines and AI-ready data platforms using Databricks, AWS cloud services, and modern data engineering frameworks. Lead end-to-end lakehouse data pipelines, ensuring performance, reliability, and cost efficiency. Champion industry best practices. Conduct and participate in peer code reviews. Identify and resolve bottlenecks in ingestion, transformation, and orchestration using Databricks Delta Live Tables, Spark optimization, and workflow automation. Implement data quality, observability, governance, and compliance using Unity Catalog, Delta Lake, and data validation frameworks. Lead technical knowledge-sharing sessions on AI/ML integration and data architecture.
Project Management:
Define project milestones, timelines, and deliverables for data and AI initiatives. Collaborate with internal and external stakeholders. Manage project priorities, sprint planning, and team workloads. Communicate risks and dependencies and develop mitigation plans to ensure on-time delivery.
Team Management and Leadership:
Supervise and mentor a team of Data Engineers (2-5). Foster continuous learning and exploration of AI and automation. Participate in recruiting, onboarding, and development activities. Conduct performance reviews and growth planning. Encourage cross-functional collaboration.
Communication:
Provide regular updates to technical and business stakeholders. Translate complex concepts for non-technical audiences. Document data engineering processes, architectures, and standards for governance and knowledge continuity.
Innovation and Other Duties:
Stay current with advancements in AI and data engineering. Pilot innovative solutions such as AI-assisted data quality checks and pipeline optimization.
Required Qualifications
Bachelor's or Master''s degree in Computer Science, Information Systems, Engineering, or equivalent experience.
5+ years designing and implementing production-grade data pipelines or enterprise-scale data platforms.
5 years of cloud-based data engineering using Databricks and AWS (e.g., Glue, S3, Lambda, Redshift).
3+ years leading teams of data/software engineers (mentorship, performance management, delivery).
Expertise in Python, PySpark, and SQL with strong data modeling and scalable transformations.
Experience with ETL/ELT across relational, non-relational, and lakehouse environments (e.g., Delta Lake, Parquet, Iceberg).
CI/CD and IaC experience (Databricks Repos, CDK, Terraform, GitHub Actions).
Knowledge of TDD and data quality frameworks; strong data governance, security, and compliance awareness.
Excellent analytical and debugging skills for distributed data systems; ability to communicate technical concepts clearly.
Experience supervising and mentoring junior team members.
Equivalent combinations of education and experience may be substituted as appropriate.
Preferred Qualifications
8+ years in Data Engineering with hands-on Databricks or similar platforms.
Experience implementing lakehouse architectures with Delta Lake, Delta Live Tables, Unity Catalog.
AI-ready data platforms and ML workflow integration (e.g., MLflow).
3+ years managing cross-functional technical teams; collaboration across Data, Analytics, and AI/ML groups.
5+ years Agile experience with JIRA or similar tools.
Expertise in distributed data processing and orchestration (Spark, Kafka, Flink, Airflow).
Familiarity with data observability, cost optimization, and performance tuning in cloud data platforms.
Professional certifications (Databricks, AWS) are desirable.
Proven ability to introduce new technologies to modernize data environments.
Commitment to continuous learning in data engineering, AI, and Databricks ecosystem.
Salary Range $125,000 - $143,712
Working Conditions
May work around standard office conditions
Keyboard use at a workstation
Use of a mouse and manual dexterity
Required Materials
Resume/CV
3 work references with contact information (at least one from a supervisor)
Letter of interest
Legal and Compliance
Employment Eligibility and E-Verify information as described in the original posting.
Equal Opportunity Employer statement and non-discrimination policy.
Background checks may be required for finalists.
#J-18808-Ljbffr
Languages
- English
Notice for Users
This job comes from a TieTalent partner platform. Click "Apply Now" to submit your application directly on their site.