Cette offre d'emploi n'est plus disponible
À propos
Location: Toronto, ON
Employment Type: Full-Time
About Cogency
Cogency is a consulting and technology services firm delivering enterprise digital transformation solutions across financial services and regulated industries. We specialize in building scalable digital platforms, automating business processes, and enabling data-driven decision-making through modern cloud and low-code technologies.
Position Overview:
We are seeking an experienced Data Engineer to design, build, and maintain scalable, high-performance data systems that support analytics, reporting, and advanced data initiatives. The ideal candidate will have strong programming expertise in Python and R, deep experience with distributed data processing using Spark and PySpark, and hands-on proficiency with modern data platforms and cloud technologies.
This role requires a strong foundation in data engineering principles, ETL development, cloud architecture, and DevOps practices to ensure the delivery of reliable, secure, and scalable data solutions.
Key Responsibilities:
Data Pipeline Development
Design, develop, and maintain scalable data pipelines using Python, R, Spark, and PySpark.
Implement batch and real-time data ingestion processes from multiple structured and unstructured data sources.
Optimize data workflows for performance, scalability, and reliability.
ETL & Data Processing
Develop, test, and optimize ETL processes to transform raw data into curated, analytics-ready datasets.
Ensure data quality, validation, and integrity across all transformation layers.
Leverage SQL extensively for querying, performance tuning, and transformation logic.
Data Storage & Management
Work with relational and non-relational databases including Postgres, MongoDB, Hive, and Trino.
Manage distributed storage systems such as HDFS and AWS S3.
Support and enhance enterprise data lake and data warehouse architectures.
Develop and maintain robust data models that support business intelligence and analytics use cases.
Cloud & Infrastructure Engineering
Build and maintain cloud-based data architectures leveraging AWS services, including S3.
Containerize applications using Docker and deploy/manage workloads using Kubernetes and OpenShift (OCP).
Ensure scalability, fault tolerance, and high availability of data systems.
Software Engineering & Best Practices
Apply software engineering principles including modular design, code reviews, unit testing, and version control.
Write clean, maintainable, and well-documented code.
Implement CI/CD pipelines to support automated testing and deployment within Agile and DevOps frameworks.
Collaboration & Delivery
Work closely with data scientists, analysts, architects, and cross-functional stakeholders to understand data requirements.
Participate in Agile ceremonies including sprint planning, stand-ups, and retrospectives.
Continuously identify opportunities for process improvements, automation, and system optimization.
Required Qualifications
Bachelor's or Master's degree in Computer Science, Engineering, Information Systems, or related field (or equivalent experience).
Strong programming experience in Python and R.
Hands-on experience with Spark and PySpark for distributed data processing.
Advanced proficiency in SQL for data manipulation and performance optimization.
Experience working with MongoDB, Postgres, Hive, Trino, HDFS, and AWS S3.
Practical experience with containerization and orchestration tools including Docker, Kubernetes, and OpenShift (OCP).
Solid understanding of ETL processes, data modeling, and data warehousing concepts.
Experience working in cloud-based environments, preferably AWS.
Familiarity with Agile and DevOps methodologies, including CI/CD pipelines.
Preferred Qualifications
Experience with large-scale enterprise data lake environments.
Knowledge of data governance, security, and compliance best practices.
Exposure to performance tuning and cost optimization in cloud environments.
Key Competencies
Strong analytical and problem-solving skills
Excellent communication and collaboration abilities
Attention to detail and commitment to data quality
Ability to manage multiple priorities in a fast-paced environment
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre a été publiée par l’un de nos partenaires. Vous pouvez consulter l’offre originale ici.