About
Job Summary We are looking for a skilled Data Engineer to design and build scalable data solutions using PySpark and AWS services. The ideal candidate will have hands-on experience in building modern data platforms using Apache Iceberg and implementing Medallion architecture on AWS.
Key Responsibilities
Design and implement end-to-end data solutions using PySpark, ensuring scalability and performance. Build and manage data pipelines using AWS services such as AWS Glue, EMR, and Lambda. Develop data products using PySpark + AWS Glue stack. Implement Medallion Architecture (Bronze, Silver, Gold layers) for structured data processing. Work with Apache Iceberg tables for efficient data storage, versioning, and schema evolution. Ensure data quality, governance, and optimization across pipelines. Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. Optimize data processing jobs and improve performance and cost-efficiency on AWS.
Required Skills & Experience
Strong experience in PySpark for data processing and pipeline development. Hands-on experience with AWS ecosystem (Glue, EMR, Lambda, S3). Experience implementing Medallion Architecture. Practical knowledge of Apache Iceberg or similar table formats. Strong understanding of distributed data processing and big data frameworks. Experience designing scalable and reliable data pipelines. Good understanding of data modeling and ETL/ELT concepts.
Preferred Qualifications
Experience working outside of Databricks-only environments (ability to build solutions using native AWS stack). Familiarity with modern data lake architectures and open table formats. Knowledge of performance tuning and cost optimization in AWS. Experience with CI/CD pipelines for data engineering workflows.
What the Client is Specifically Looking For
Engineers who can independently design solutions using PySpark (not limited to Databricks). Strong expertise in AWS-native data engineering tools. Hands-on implementation experience with Apache Iceberg (preferred over Delta). Ability to build data products using Glue + PySpark stack. Clear understanding and implementation of Medallion architecture using AWS services.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Languages
- English
Notice for Users
This job comes from a TieTalent partner platform. Click "Apply Now" to submit your application directly on their site.