Global IT LLM Engineer Senior Manager

We are seeking a skilled Generative AI Engineer to join our Legal Digital Products team. This role will involve designing, developing, deploying, and optimizing Gen AI products at scale.

• Conceptualize Generative AI custom products at BCG internally and operate as a technical thought partner for digital leaders and product owners.
• Combine data science and engineering expertise in building Generative AI products.
• Develop Gen AI products as individual contributors to develop scalable high-performance enterprise products along with broader squad members.
• Share expertise and recommendations on alternate approaches and tools to improve products.
• Collaborate closely with data engineers, data scientists, software engineers, QA, product owners, and analysts to develop seamless end-to-end integrated products.
• Build strong relationships with business stakeholders, ensuring alignment on project goals and successful integration of products into business processes.
• Oversee and mentor other LLM/AI engineers, data scientists in the project on Gen AI expertise.
• Share Gen AI knowledge and expertise across with broader digital teams squads, chapters, CoEs.
• Regularly ramp up on the latest LLMs, technologies, frameworks, and platforms in the Gen AI space.

• Overall 9+ years of experience in engineering, machine learning & LLMs.
• 6+ years of hands-on experience in building and deploying AI/ML/Gen AI solutions in large business-critical applications.
• 2+ years of experience in developing Gen AI products using various LLMs and frameworks.
• Advanced proficiency in Python for data science, engineering, and LLMs.
• Highly curious technically to evaluate and test new technologies in the Gen AI space.
• Proven ability to work with large structured and unstructured datasets and deploy AI/ML/Gen AI solutions through automated pipelines.
• Experience with leveraging, training, and fine-tuning Foundation Models including multimodal inputs and outputs.
• Strong experience working with key LLM models APIs (e.g. OpenAI, Anthropic) and LLM Frameworks (e.g. LangChain, LlamaIndex).
• Proficiency in generating and working with embeddings across various data formats. Understanding of embedding spaces and their applications in semantic search and information retrieval.
• Knowledge of effective text chunking techniques for optimal processing and indexing of large documents or datasets.
• Expertise with RAG concepts and fundamentals (vectorDBs, semanticsearch, re-rankers etc.), Expertise in implementing RAG systems that combine knowledge bases.
• Experience with multi-agent frameworks/systems and an understanding of multi-agent systems and their applications in complex problem-solving scenarios.
• Experience in working with various databases (SQL, no SQL), APIs, and microservices development.
• Experience with LLMOps tools (e.g. Langsmith) to implement guardrails, track accuracy, hallucinations, bias, and other metrics in Gen AI products.
• Experience in constructing and querying knowledge graphs including graph-based reasoning.
• Experience in front-end development technologies (e.g. REACT) would be good to have.
• Excellent communication skills, with the ability to explain complex technical concepts to various audiences.
• Proven ability to operate with a transparent mindset, communicating openly with stakeholders at various levels of the organization.