Data Engineer II NRG Energy, Inc. Location: Utah Category: DataElectricenergyonline • United States
This job offer is no longer available
Data Engineer II NRG Energy, Inc. Location: Utah Category: Data
Electricenergyonline
- +1
- +5
- United States
- +1
- +5
- United States
About
In this role, you will help drive Vivint's data platform. You will work with a talented team of data scientists and engineers to provide insights across the organization. As a Data Engineer, you will be instrumental in designing, building, and maintaining robust datasets and ETL (Extract, Transform, Load) pipelines that process and integrate large volumes of IoT (Internet of Things) data. Your work will empower business users by delivering high-quality, actionable insights and facilitating data-driven decision-making across the organization. Primary Responsibilities
Design and Develop ETL Pipelines: Architect, implement, and optimize ETL processes to handle the ingestion, transformation, and storage of large-scale IoT data streams. Ensure data accuracy, consistency, and availability for analysis. Create and Maintain Datasets: Develop and manage comprehensive datasets that aggregate IoT data from various sources. Ensure datasets are well-structured, accessible, and aligned with business needs. Data Integration and Optimization: Integrate disparate data sources and optimize data storage and retrieval processes to support efficient querying and reporting. Implement best practices for data quality and performance. Collaborate with Stakeholders: Work closely with business users, data scientists, and analysts to understand data requirements and translate them into technical solutions. Provide support and troubleshooting for data-related issues. Monitor and Improve Data Systems: Continuously monitor the performance of data pipelines and datasets. Identify and address potential issues, and implement improvements to enhance efficiency and reliability. Documentation and Compliance: Document data processes, pipeline architectures, and system changes. Ensure compliance with data governance and security policies. Required Skills
Minimum of 4 years of professional experience in data analysis or data engineering Experience with Big Data/Hadoop applications. Experience with Spark, Hive, Kafka, and Map Reduce SQL experience and strong background in optimizing SQL queries for transforming data Experience with different data formats (Parquet, Arvo, Json, XML, etc.) Experience bringing in data from multiple Data Sources / API's Building and Debugging Data Pipelines Distributed systems (pipelines and databases) Proficient in Python development for data pipelines and automation, including debugging, testing, and handling development. At least 2 years experience preferred. Experience in Linux environments, batch automation, and shell scripts Nice To Have
Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and data warehousing solutions (e.g., Redshift, Snowflake) Required Education/Experience
Graduate Degree in a Quantitative field (Math, Statistics, Engineering, Computer Science) or relevant experience. 3+ years work experience. Learn about the Vivint Culture and why it's a great place to grow your career! Read the full posting. #J-18808-Ljbffr
Nice-to-have skills
- Hadoop
- Hive
- Kafka
- SQL
- Spark
Work experience
- Data Engineer
Languages
- English
Notice for Users
This job was posted by one of our partners. You can view the original job source here.