Cette offre d'emploi n'est plus disponible
À propos
Data Scientist
At Zywave, we believe in building AI systems that are reliable, measurable, and continuously improving. The Data Scientist will work within our AI Engineering and Data Science team to execute and operationalize our evaluation framework for ML/AI models. This role is hands-on and execution-focused, requiring strong technical skills in statistical analysis, and production ML workflows. You'll implement evaluation pipelines, run experiments, and conduct analyses that ensure our AI systems meet quality standards and drive measurable business impact.
What you will do:
Evaluation Framework Execution Implement and operationalize evaluation pipelines for agentic AI models based on established frameworks and methodologies. Execute benchmark tests, collect performance metrics, and generate evaluation reports. Maintain and improve automated testing suites that assess model performance across multiple dimensions. Monitor evaluation results and flag performance issues or anomalies for investigation. Experimentation & A/B Testing
Execute A/B tests and experiments according to designed protocols, ensuring proper implementation and data collection. Conduct statistical analysis of experiment results, including hypothesis testing, confidence intervals, and effect size calculations. Create clear, actionable reports that communicate experiment findings to stakeholders. Support the design of new experiments by providing data-driven insights and feasibility assessments. Traditional ML & Statistical Analysis
Build and maintain classical machine learning models to support business analytics and decision-making. Conduct exploratory data analysis to uncover insights and patterns in product usage and model performance data. Perform statistical analyses including regression analysis, time series forecasting, and cohort analysis. Develop predictive models for business metrics, user behavior, and model performance trends. Data Workflows & Pipeline Management
Build and maintain data pipelines that support evaluation workflows, analytics, and reporting. Ensure data quality through validation checks, monitoring, and documentation. Work with data engineering teams to optimize data collection and storage for evaluation needs. Create and maintain SQL queries, transformations, and data models. Monitoring & Reporting
Develop and maintain dashboards that track model performance, experiment results, and key metrics. Create automated reporting that provides visibility into AI system health and business impact. Support incident response by analyzing data to identify root causes of performance issues. Document processes, methodologies, and findings to build organizational knowledge. What you should bring:
Strong foundation in statistics and experimental design-you understand A/B testing, hypothesis testing, and can properly analyze experiment results. Hands-on experience with machine learning fundamentals, including model training, evaluation, and feature engineering. Proficiency in Python and the data science stack (pandas, scikit-learn, NumPy, matplotlib/seaborn). Strong SQL skills and experience working with relational databases and data warehouses. Experience building data pipelines or working with workflow orchestration tools. Familiarity with version control (Git) and collaborative software development practices. Strong analytical and problem-solving skills with attention to detail. Good communication skills-you can explain technical concepts clearly and create compelling visualizations. Self-motivated with the ability to manage multiple tasks and priorities effectively. (Preferred) Exposure to generative AI, LLM evaluation, or working with AI/ML products in production. (Preferred) Experience with data visualization tools (Tableau, Looker, or similar).
Compétences linguistiques
- English
Avis aux utilisateurs
Cette offre a été publiée par l’un de nos partenaires. Vous pouvez consulter l’offre originale ici.