À propos
Must Have Technical/Functional Skills: Experience: Typically requires 5+ years of experience in data engineering or a related role, with a proven track record of deploying and managing large-scale distributed systems. Programming Languages: Strong proficiency in Python (including PySpark) and SQL is essential, with additional experience in Java or Scala being a plus. Big Data Technologies: Expertise in the Hadoop ecosystem and components, along with distributed computing frameworks like Apache Spark and Kafka, is crucial. Databases & Cloud Platforms: Experience with relational (e.g., PostgreSQL, MySQL) and NoSQL databases, and familiarity with cloud services (AWS, GCP, or Azure). Problem-Solving: High-level analytical and problem-solving skills to resolve complex technical data issues. Soft Skills: Excellent written and verbal communication skills, a team-player attitude, and the ability to manage multiple initiatives are vital for success in this role. Roles & Responsibilities:
Data Pipeline Development: Design, build, and maintain scalable ETL/ELT processes and data pipelines using Python, SQL, and big data technologies (Hadoop, Spark, Hive, Kafka). Big Data Management: Work within the Hadoop technology stack, including HDFS, Hive, Yarn, Impala, and HBase, to manage and store large datasets. Performance Optimization & Automation: Troubleshoot, tune, and optimize data processing jobs and database performance, while identifying opportunities for automation in testing and deployment processes (CI/CD). Architecture & Design: Lead in developing data solutions and designing data service infrastructure, contributing to overall data architecture decisions. Collaboration & Mentorship: Collaborate with data scientists, analysts, and business stakeholders to understand data requirements, and provide technical guidance and mentorship to junior team members. Data Quality & Governance: Ensure data accuracy, integrity, and security by implementing validation checks and adhering to data governance standards.
Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.