À propos
We are on the lookout for a talented Data Engineer with substantial experience in software development, specializing in OpenSearch and data analysis. In this vital role, you will play a key part in creating and supporting high-quality data pipelines and search indexes for our cutting-edge Cloud Inventory Solution. Your expertise in ingesting data from Oracle Autonomous Database and Kafka into OpenSearch will ensure top-notch data accuracy, providing exceptional search and analytical capabilities for our diverse stakeholders. Your Impactful Responsibilities Data Engineering (Streaming + Batch) Design, implement, and manage robust, production-grade data pipelines to seamlessly ingest cloud inventory data from Oracle Autonomous Database and Kafka into OpenSearch. Establish efficient processing patterns for updates, deduplication, and recovery to ensure data integrity and reliability. Collaborate effectively with cross-functional teams on data contracts and schema management, fostering seamless integration of data systems. OpenSearch Development Create and maintain insightful OpenSearch index mappings and templates to cater to diverse inventory search and filtering requirements. Develop, test, and optimize complex OpenSearch queries and aggregations, delivering valuable insights and reporting capabilities. Address indexing and querying challenges to enhance system performance continually. Data Analysis & Quality Assurance Perform thorough data profiling and analysis to maintain data integrity between source and indexed systems. Establish and execute data quality checks, promptly informing stakeholders of any discrepancies. Support incident resolution and root-cause analysis to mitigate future data-related issues. AI / ML Enablement Contribute to advancing AI capabilities, focusing on features like entity extraction and anomaly detection using our diverse datasets. Assist in preparing datasets for AI applications and implementing monitoring processes. Collaborate with machine learning engineers to integrate model outputs seamlessly into our data pipelines. Operational Excellence & Compliance Support the maintenance of monitoring and alerting systems to ensure the health and efficiency of our data pipelines. Engage in on-call support and develop solutions to enhance operational readiness. Ensure adherence to security and compliance practices for responsible management of sensitive data. Required Qualifications 5+ years of experience in data engineering or related backend development. Strong expertise in OpenSearch/Elasticsearch development with comprehensive knowledge of index design and query optimization. Experience with Kafka for effective streaming data processing. Exceptional SQL skills along with a solid understanding of relational databases, particularly Oracle. Proven aptitude in data analysis and a knack for communicating insights effectively. Hands-on experience in AI/ML projects, preferably using Python. Preferred Qualifications Familiarity with Change Data Capture (CDC) patterns and data integration tools. Experience with cloud services and operations. Understanding of vector search concepts and data quality tools. Enjoy a competitive salary and an impressive benefits package including medical, dental, and vision coverage. We invite you to apply and become part of a team that is shaping the future of AI and cloud solutions!
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre provient d’une plateforme partenaire de TieTalent. Cliquez sur « Postuler maintenant » pour soumettre votre candidature directement sur leur site.