Principal Data Scientist - Agent BuilderThe Consulting Solutions • San Antonio, Colorado, United States

Jetzt Bewerben

Principal Data Scientist - Agent Builder

The Consulting Solutions

San Antonio, Colorado, United States

San Antonio, Colorado, United States

Jetzt Bewerben

Über

Company Overview Elastic, the Search AI Company, enables everyone to find the answers they need in real time, using all their data, at scale — unleashing the potential of businesses and people. The Elastic Search AI Platform, used by more than 50% of the Fortune 500, brings together the precision of search and the intelligence of AI to enable everyone to accelerate the results that matter. Overview The Search Conversational Experiences team builds Elastic’s new conversational and agentic platform that lets customers chat with their own data in Elasticsearch. As a Principal Data Scientist you will set the technical direction for evaluating, improving, and scaling chat quality across Elastic’s agentic platform, defining evaluation strategy, guiding product decisions, and working closely with backend engineering, product, UX, and other data scientists. Responsibilities Define the evaluation strategy for conversational and agentic search, including offline and online evaluation, golden datasets, rubrics, LLM-as-judge calibration, groundedness and citation checks, and A/B testing. Lead the design of quality metrics and decision frameworks for RAG, agents, tools, model selection, agent routing, prompt behavior, and cost/latency trade‑offs. Build, compare, and guide improvements across retrieval and re‑ranking approaches, including sparse and dense retrieval, vector search, query understanding, semantic rewrites, and context enrichment. Turn experimental results into product and business decisions: which models to use, how to route requests efficiently, which tools should be exposed, and how agents should be customized for different Elastic use cases. Partner with engineering to productionize evaluation pipelines, telemetry, dashboards, CI guardrails, and regression detection for chat quality, helpfulness, dedication, latency, and cost. Influence the roadmap by identifying the highest‑leverage quality gaps, proposing practical solutions, and communicating trade‑offs clearly to product, engineering, and leadership. Mentor other data scientists and engineers in experiment design, evaluation methodology, statistical rigor, and practical approaches to improving LLM‑powered systems. Share outcomes through clear docs, notebooks, PRs, dashboards, technical proposals, and cross‑functional reviews. Qualifications 8+ years of applied DS/ML experience, with deep expertise in IR, NLP, ranking, semantic search, RAG, or LLM‑powered product experiences. Strong track record defining and leading evaluation for production AI/ML systems, including offline metrics, online experimentation, LLM‑as‑judge approaches, groundedness, citation quality, and model comparison. Experience influencing product and technical strategy through data, especially in ambiguous or emerging domains where the “right” metric or approach is not obvious at the start. Hands‑on ability with Python, PyTorch/Transformers, Pandas, notebooks, reproducible experiments, versioned datasets, and clean, reviewable code. Strong understanding of retrieval systems, including dense and sparse retrieval, re‑ranking, vector search, query understanding, and evaluation metrics such as nDCG, MRR, Recall@k, precision, and latency/cost trade‑offs. Experience collaborating closely with engineering teams to move from prototype to production, including telemetry design, dashboards, CI guardrails, and quality regression tracking. Practical Elasticsearch experience, or experience with similar search and distributed data systems. ES|QL familiarity is a plus. Excellent written and verbal communication, with the ability to explain complex scientific and technical trade‑offs to engineering, product, design, and leadership audiences. A collaborative, low‑ego style and a strong ability to mentor, raise standards, and develop transparency for others in a distributed team. Compensation Base salary only. Salary range: €73,300 – €115,900 EUR. No variable compensation component. Benefits Health coverage for you and your family in many locations. Flexible scheduling and remote work options. Generous vacation days annually. Parental leave of at least 16 weeks. Volunteer time: up to 40 hours per year. Employee donation matching up to €2,000. EEO Statement Elastic is an equal opportunity employer and is committed to creating an inclusive culture that celebrates different perspectives, experiences, and backgrounds. Qualified applicants will receive consideration for employment without regard to race, ethnicity, color, religion, sex, pregnancy, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, disability status, or any other basis protected by federal, state or local law, ordinance or regulation. We welcome individuals with disabilities and strive to create an accessible and inclusive experience for all individuals. To request an accommodation during the application or recruiting process, please email candidate_accessibility@elastic.co. Export Controls Elasticsearch develops and distributes technology subject to U.S. and other countries’ export controls. Applicants located in or nationals of Belarus, Cuba, Iran, North Korea, Syria, or Russia, including the Ukrainian territories annexed by Russia, may require an export license to be employed in this role. National origin or nationality does not affect eligibility for employment with Elastic.
#J-18808-Ljbffr

San Antonio, Colorado, United States

Sprachkenntnisse

English

Hinweis für Nutzer

Dieses Stellenangebot stammt von einer Partnerplattform von TieTalent. Klick auf „Jetzt Bewerben”, um deine Bewerbung direkt auf deren Website einzureichen.

Jetzt Bewerben