About
The following information aims to provide potential candidates with a better understanding of the requirements for this role.
We are seeking several Snowflake Data Engineers - both a Senior Data Engineer and a Junior Data Engineer to join a dynamic data team within the Energy sector. These roles are integral to designing, developing and maintaining scalable data pipelines and architectures that support advanced analytics, machine learning and business intelligence initiatives.
The ideal candidates will possess strong expertise in cloud platforms, big data technologies and data modeling, contributing to innovative data solutions that drive strategic decision making across the organisation.
Data Engineers will ensure that the pipelines are developed and deployed with a secure by design approach, delivering robust, thoroughly tested and maintainable solutions.
* Develop piplines using the standard patterns for data pipelines and workflows utilizing Streamsets, Kestra, dbt, Git. * Design and Implement data storage and processing solutions employing Snowflake. * Utilize AWS services for cloud-based platform tooling infrastructure including but not limited to: Lambda,ECS,MSK,RDS,EC2, Secrets Manager, ALB, Cloud Watch, Event Bridge * Utilize Terraform for AWS and Azure deployments. * Leverage and integrate APIs for data access and manipulation. * Write Python scripts for data common processing and automation tasks. * Leverage Platform API's and Web Applications to enforce Platform Security * Development experience with Go, SQL, C#, .net, JavaScript, shell scripts & container platforms like Docker. * The engineer shall have experience in Authentication mechanisms including but not limited to (OAuth 2.0, OIDC, Microsoft Entra, Key Pair Authentication, Certificate based authentication, SAML based SSO).
Additionally, there is a requirement to create and execute comprehensive test plans to ensure the pipelines functionality and performance as well as develop unit tests, integration tests and end-to-end tests for data pipelines and workflows. Governance requires adherence to access processes and procedures to maintain Data Privacy and Security.
Platform Maintenance is also required to monitor and manage the platform to ensure optimal performance and uptime as well as conduct regular maintenance tasks such as updates, patches and backups. Security responsibilities require best practices throughout the pipeline development and deployment process. Therefore, conduct regular security reviews and vulnerability assessments.
Deliverables
* Robust and scalable data pipelines with well-documented code and processes. * Comprehensive test plans and automation scripts ensuring platform reliability. * Regular security assessments and compliance reports.
Expectations
* Deliver secure, robust, and maintainable data pipelines. * Ensure high-quality and thoroughly tested data solutions. * Maintain compliance with security standards and best practices. * Maintain compliance with the Data Lifecycle Management Process, Data Privacy standards and best practices.
Initial 6-month contract with the opportunity to extend. 1099 contract preferred. xawqmyg Houston based.
Work Location: Hybrid remote in Houston, TX 77056
Job Type: Contract
Pay: $450.00 - $960.00 per day
Work Location: Hybrid remote in Houston, TX 77056
Languages
- English
Notice for Users
This job comes from a TieTalent partner platform. Click "Apply Now" to submit your application directly on their site.