About
Key Responsibilities: Designs and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured). Designs and implements framework to continuously monitor and troubleshoot data quality and data integrity issues. Implements data governance processes and methods for managing metadata, access, retention to data for internal and external users. Designs and provide guidance on building reliable, efficient, scalable and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages. Designs and implements physical data models to define the database structure. Optimizing database performance through efficient indexing and table relationships. Participates in optimizing, testing, and troubleshooting of data pipelines. Designs, develops and operates large scale data storage and processing solutions using different distributed and cloud based platforms for storing data (e.g. Data Lakes, Hadoop, Hbase, Cassandra, MongoDB, Accumulo, DynamoDB, others). Uses innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity. Assists with renovating the data management infrastructure to drive automation in data integration and management. Ensures the timeliness and success of critical analytics initiatives by using agile development technologies such as DevOps, Scrum, Kanban Coaches and develops less experienced team members Skillset: Hands-on Azure Data experience Tools to have: Databricks, Synapse, Data Factory, etc. High functional knowledge of Big Data Hands-on experience automating the ETL process Scala or PySpark knowledge (Scala preferred) Required Skills: -strong Azure Data Services -MUST HAVE Azure API development experience -Scala or PySpark -strong SQL knowledge Basic Qualification: Additional Skills: -100% remote -long term contract-to-hire Background Check: Yes Notes: If you are going to use a subvendor, please make sure to include/notify me of the direct employers name so I can check to see if that employer is in good standing with Apex Systems. I will be very slammed today (3.10) and may not be able to get to candidates until Fri 3.11.
Languages
- English
Notice for Users
This job comes from a TieTalent partner platform. Click "Apply Now" to submit your application directly on their site.