Big Data Engineer Senior
SAIC
- Arlington, Texas, United States
- Arlington, Texas, United States
Über
Positions are contingent pending contract award.
Place of Performance: The work will be performed in Arlington, Virginia. Some work may be performed remotely, subject to Government approval.
Job Responsibilities
Design, build, and maintain the core WDP data infrastructure and pipelines using agentic workflows to streamline operations and enhance automation.
Ensure the scalability, reliability, and security of the platform to support data ingestion, processing, storage, and transformation for various consumers, including data tooling teams.
Build and operate data pipelines and Lakehouse solutions leveraging AWS (e.g., S3, Glue, Lake Formation, EMR, Kinesis, Redshift, Athena, Lambda, IAM) and Databricks (e.g., Delta Lake, Unity Catalog, Delta Live Tables, Apache Spark) within an AWS GovCloud environment.
Develop enterprise data products accessible via a modern data marketplace, implement a medallion architecture for data storage (bronze, silver, gold zones), and establish self-service inbound and outbound data connection methods.
Implement data quality monitoring tools, proactive alerting for pipeline issues, and data drift detection mechanisms.
Enforce data security policies, optimize infrastructure for performance and scalability, ensure data backup and recovery, and implement robust data governance policies.
Document pipelines, models, and governance standards while ensuring compliance with data privacy and security regulations.
Design and develop solutions to complex applications problems, system administration issues, or network concerns.
Perform systems management and integration functions.
Qualifications
Bachelors and nine (9) years or more experience; Masters and seven (7) years or more experience ; PhD or JD and four (4) years or more experience.
TS/SCI Clearance is required.
Required Certification: CompTIA Security+ (CE).
Knowledge, Skills, Abilities, and Competencies
Data Architecture & Engineering
Expertise in data architecture and medallion data storage frameworks (bronze, silver, gold zones).
Proficiency in developing and managing enterprise data products and marketplaces.
Hands‑on experience designing and operating enterprise data platforms using AWS (S3, Glue, Lake Formation, EMR, Redshift, Kinesis, Athena, Lambda, IAM) for data lake/medallion storage, ingestion, ETL, and governance.
Proven experience building and managing data pipelines on the Databricks Lakehouse Platform , including Delta Lake (medallion architecture), Unity Catalog (governance, lineage, access control), and Delta Live Tables for declarative pipeline development and data quality enforcement.
Proficiency with Apache Spark (PySpark/Scala/SQL) for large‑scale data processing and transformation.
Data Quality & Governance
Ability to implement and maintain data quality monitoring tools, including proactive alerting and data drift detection (e.g., DLT expectations, Lakehouse Monitoring).
Knowledge of data backup, recovery, and robust governance protocols.
Skill in documenting data pipelines, models, and governance standards.
Familiarity with compliance requirements for data privacy and security regulations.
Security & Infrastructure
Strong understanding of data security policies and practices, including encryption (KMS), least‑privilege IAM, and secure networking constructs.
Familiarity with deploying and operating workloads within AWS GovCloud (US) and DoD security boundaries (IL4–IL6).
Experience optimizing infrastructure for performance and scalability.
Familiarity with Infrastructure as Code (IaC) using Terraform and/or AWS CloudFormation for repeatable, secure provisioning.
Soft Skills
Effective communication and collaboration skills for cross‑functional teamwork.
Analytical and problem‑solving abilities for troubleshooting and resolving technical challenges.
Preferred Certifications
The following certifications are considered a plus:
AWS
AWS Certified Data Engineer – Associate.
AWS Certified Solutions Architect – Associate or Professional.
AWS Certified Security – Specialty.
Databricks
Databricks Certified Data Engineer Associate.
Databricks Certified Data Engineer Professional.
Databricks Certified Apache Spark Developer.
REQNUMBER: 2613397
EOE AA M/F/Vet/Disability
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.