Data Engineer - AWS/Databricks - Mid Level
Acuity
- Reston, Virginia, United States
- Reston, Virginia, United States
Über
Responsibilities
Build and maintain scalable PySpark‑based data pipelines in Databricks notebooks to support ingestion, transformation, and enrichment of structured and semi‑structured data.
Design and implement Delta Lake tables optimized for ACID compliance, partition pruning, schema enforcement, and query performance across large datasets.
Develop ETL and ELT workflows that integrate multiple source systems into a centralized, query‑optimized data warehouse architecture.
Leverage Spark SQL and DataFrame APIs to implement business rules, dimensional joins, and aggregation logic aligned to warehouse modeling best practices.
Collaborate with data architects and engineers to implement cloud‑native data solutions on AWS using S3, Glue, RDS, and IAM for secure, scalable storage and access control.
Optimize pipeline performance through intelligent partitioning, caching, broadcast joins, and adaptive query tuning.
Deploy and version data engineering assets using Git‑integrated development workflows and automate deployment with CI/CD tools such as GitLab or Jenkins.
Monitor pipeline health, job execution, and cluster utilization using native Databricks tools and AWS CloudWatch, identifying bottlenecks and optimizing cost‑performance tradeoffs.
Conduct technical discovery and mapping of legacy source systems, identifying required transformations and designing end‑to‑end data flows.
Implement governance practices including metadata tagging, data quality validation, audit logging, and lineage tracking using platform‑native features and custom logic.
Support ad hoc data access requests, develop reusable data assets, and maintain shared notebooks that meet operational reporting and analytics needs across teams.
Qualifications
4+ years of experience in data engineering and Agile analytics.
4+ years of experience creating software for retrieving, parsing and processing structured and unstructured data.
2+ years of experience building scalable ETL and ELT workflows for reporting and analytics.
2+ years experience building enterprise data engineering solutions in the cloud, with preferred experience with cloud native technologies from AWS and Databricks.
Experience with data quality, validation frameworks, and storage optimization strategies.
BA or BS degree.
Clearance Requirement
Must be US Citizen with an ability to obtain and maintain US Suitability.
Benefits
Grow Your Career, Your Way – Personalized development plans, mentorship, and up to $3,000 annually for training and certifications and up to $3,000 for degree‑seeking programs.
Be Part of Something Innovative – Work on cutting‑edge solutions that support important government missions in an environment encouraging new ideas and continuous improvement.
Thrive in a People‑First Culture – Collaboration, respect, and support are how we operate; your voice is heard, contributions are recognized, and success is shared.
Feel Valued and Rewarded – Competitive compensation, comprehensive benefits, and a strong focus on work‑life balance.
Join an Award‑Winning Team – Employee rank among best employers, earning honors like Best Places to Work and Top Workplaces.
Bring Your Whole Self to Work – Commitment to building a diverse, inclusive environment where everyone feels respected, supported, and empowered to succeed.
Make Your Impact – Your ideas are valued, growth is supported, and work drives meaningful outcomes.
Acuity is an Equal Opportunity Employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.