Lead analysis and improvement of conversational AI systems by designing prompt strategies, optimizing RAG and retrieval, evaluating LLMs, defining AI quality metrics, running experiments, and collaborating with engineers to deploy production-scale improvements.
About the role
We are looking for a Senior AI Scientist to continuously improve the quality, accuracy and effectiveness of our AI-powered products. You will analyse AI interactions, identify improvement opportunities and optimise conversational behaviour, prompting strategies and retrieval mechanisms. Working closely with AI Engineers, you will drive measurable improvements in AI performance and user satisfaction.
What you'll be doing
- Analyse AI conversations and identify opportunities to improve response quality and user experience.
- Design, implement and evaluate prompt engineering strategies.
- Optimise RAG pipelines and retrieval quality.
- Evaluate different LLMs and recommend the most effective models for specific use cases. Potentially train customer personal models.
- Define and monitor AI quality metrics, including accuracy, relevance, latency and user satisfaction.
- Design and execute experiments to validate AI improvements.
- Analyse user feedback and interaction data to continuously improve AI behaviour.
- Collaborate with AI Engineers to deploy validated improvements into production.
- Maintain evaluation frameworks, benchmarks and documentation.
- Stay current with advances in generative AI and recommend practical improvements to existing products.
What you'll bring
- 5+ years of experience in AI, ML, NLP or a related field.
- Strong understanding of LLMs and conversational AI.
- Experience with prompt engineering, prompt evaluation and LLM optimisation.
- Experience evaluating conversational AI systems using qualitative and quantitative methods.
- Strong Python and SQL skills.
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management skills.
- Proactive mindset with strong ownership and accountability.
- Experience with speech and voice AI technologies.
- Experience with AI evaluation frameworks and benchmarking.
- Experience with vector databases and embedding models.
- Knowledge of AI governance, safety and responsible AI practices.
- Experience working with production AI systems at scale.
What's in it for you
- Sporty is a remote first company in pursuit of sustainability
- A competitive salary + individual performance based bonuses every quarter
- 28 days paid annual leave
- Our core working hours are 10am-3pm in your local time zone with flexibility outside of this
- Referral bonuses & flash bonuses
- Top of the line equipment
- Annual company retreats to provide great internal networking opportunities
Interview Process
- Remote video screening with our Talent Acquisition Team
- Online home assignment
- Remote video interview with Team Members (3x45 Mins)
If you're interested, we encourage you to apply! Every application is reviewed by a member of our team (AI is not used in our recruitment process), and we aim to respond within 48 hours.
Similar Jobs
Artificial Intelligence • Machine Learning • Natural Language Processing • Software • Conversational AI
The Account Executive will drive new customer acquisition and revenue, self-prospecting to build sales pipelines, and collaborating with marketing and sales teams. Responsibilities include understanding customer needs in voice AI, articulating Deepgram's value, and managing existing accounts for upsell opportunities.
Top Skills:
AIAPIsMlSpeech-To-SpeechSpeech-To-TextText-To-Speech
Artificial Intelligence • Productivity • Software • Automation
As a Sr. Applied AI Engineer at Zapier, you will build and enhance AI platform capabilities, focusing on LLM Ops and ML Ops to support scalable AI development across teams.
Top Skills:
Cloud InfrastructureLlm OpsMl OpsPythonTypescript
.png)
Artificial Intelligence • Blockchain • Fintech • Financial Services • Cryptocurrency • NFT • Web3
Serve as Finance's single DRI for product launches with financial impact: evaluate new trade strategies, document fund flows and accounting treatments, design and implement financial controls, coordinate cross-functional stakeholders (accounting, tax, treasury, legal, operations), and partner with IT/vendors to automate and scale workflows for audit and regulatory readiness.
Top Skills:
Generative Ai
What you need to know about the Bristol Tech Scene
Along with Gloucester, Swindon and Bath, Bristol is part of the "Silicon Gorge" tech hub, a region in the U.K. renowned for its high-tech and research-driven industries, with a particular emphasis on sustainability and reducing environmental impact. As the European Green Capital, Bristol is home to 25,000 cleantech companies, including Baker Hughes and unicorn Ovo Energy. The city has committed to achieving net-zero emissions within the next decade.
