Sr. Data Engineer
Sciata
- New York, New York, United States
- New York, New York, United States
Über
Primary Responsibilities & Essential Functions
Design, implement, and maintain the data pipeline architecture for our business
Assemble large, complex data sets that meet functional / non-functional business requirements
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re‑designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and GCP or AWS “big data” technologies
Build analytics tools that utilize the data pipeline to provide actionable insights into sensor behavior, application usage, operational efficiencies, and other key business performance metrics
Work across vertical stakeholders to assist with data‑related technical issues as well as supporting their evolving data infrastructure needs
Develop and implement a data segregation and security strategy within and across availability zones
Create the data tools and frameworks for our analytics and data science team that assist them in building industry‑leading insights and solutions
Skills & Qualifications Minimum
5+ years of experience in a data engineer (or similar) role
Advanced experience working with both relational & non‑relational databases, including Postgres, BigQuery, and Firebase
Experience with big data tools: Hadoop, Spark, Kafka, etc.
Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
Experience with stream‑processing systems: Storm, Spark‑Streaming, etc.
Experience with object‑oriented/object function scripting languages: Python, Kotlin, etc.
Experience with machine learning frameworks (like Keras or PyTorch) and libraries (like scikit‑learn)
Advanced working SQL knowledge and query authoring
Experience building and optimizing “big data” data pipelines, architectures and data sets
Strong track record of manipulating, processing and extracting value from large, disconnected datasets
Working knowledge of message queuing, stream processing, and highly scalable “big data” data stores
Strong project management and organizational skills
Experience supporting and working with cross‑functional teams in a dynamic environment
Preferred
Experience with GCP cloud services: BigQuery, Firebase
Experience working with IoT sensors and their data sets
Experience working with data visualization tools, especially Power BI
#J-18808-Ljbffr
Sprachkenntnisse
- English
Hinweis für Nutzer
Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.