This job offer is no longer available
About
Onsite from Day 1
Required Skill:
Design and build robust, scalable ETL/ELT pipelines using PySpark to ingest data from diverse sources (databases, logs, APIs, files). Transform and curate raw transactional and log data into analysis-ready datasets in the Data Hub and analytical data marts. Develop reusable and parameterized Spark jobs for batch and micro-batch processing. Optimize performance and scalability of PySpark jobs across large data volumes. Ensure data quality, consistency, lineage, and proper documentation across ingestion flows. Collaborate with Data Architects, Modelers, and Data Scientists to implement ingestion logic aligned with business needs. Work with cloud-based data platforms (e.g., AWS S3, Glue, EMR, Redshift) for data movement and storage. Support version control, CICD, and infrastructure-as-code where applicable. Participate in Agile ceremonies and contribute to sprint planning, story grooming, and demos.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.