Service Desk Night Analyst joining King’s Service Centre. Providing exceptional IT and Facilities Management support to staff and students at King's College London during night hours.
About the role
- GenAI Tuning Analyst responsible for AI performance in Contact Center. Collaborating across teams to optimize AI deployment and customer interaction quality.
Responsibilities
- Lead Reinforcement Learning from Human Feedback (RLHF) cycles.
- Review AI-generated outputs and 'grade' them based on accuracy, empathy, and adherence to compliance protocols.
- Define and track AI-specific KPIs, such as Hallucination Rate, Refusal Rate, and Intent Accuracy, alongside traditional metrics like First Contact Resolution (FCR).
- Implement and monitor safety filters to ensure the AI remains neutral, avoids prohibited topics, and protects PII (Personally Identifiable Information).
- Partner with Subject Matter Experts (SMEs) in Legal, Compliance, and Operations to translate business policy into technical model constraints.
- Iterate on complex system prompts to guide LLM behavior for specific contact center use cases (e.g., automated summaries, live chat responses, and knowledge base retrieval).
- Identify and curate high-quality 'golden datasets' from historical call transcripts and chat logs to be used for fine-tuning and few-shot prompting.
Requirements
- 2+ years in AI/NLP, Data Analysis, high-level Contact Center Operations.
- Analyzing Intent Accuracy to identify gaps in the AI’s natural language understanding (NLU/NLP).
- UAT Support: Experience assisting in User Acceptance Testing for new AI features or tool rollouts.
- Secondary experience in Quality Assurance is a plus.
- Proficiency in prompt engineering techniques (Chain-of-Thought, Few-Shot, Zero Shot).
- Basic understanding of LLM architectures (GPT-4, Claude, Llama).
- Exceptional written and verbal communication skills with an obsession over tone, grammar, and brand consistency.
- Ability to act as a bridge between technical teams and frontline operations.
- Moderate customer meetings to align with company direction on our AI tools.
Benefits
- Healthcare benefits
- 401K plan
- Paid company holidays
- Paid vacation
- Business casual work environment
- Annual performance based bonus program
Job title
Job type
Experience level
Salary
Not specified
Degree requirement
Location requirements
Analista Funcional Jr focused on user - centric solutions for financial software at a top fintech. Engaging with clients and ensuring product functionality results in successful user experience.
Senior Analyst supporting Business Resiliency team in advancing enterprise resilience strategy. Leading key components of business continuity, cyber resilience, and crisis management programs.
Senior Analyst in Business Analysis and Insights evaluating financial data for strategic decisions at Walmart. Collaborating with partners for consistent reporting and actionable recommendations.
PMO Analyst providing project management support for rigorous risk transformation initiatives at Mercer, based in Melbourne or Sydney.
Analyst providing specialist advice in quantitative analytics and risk management solutions at Absa Group. Focusing on ESG & Climate Risk Measurement solutions across the group.
Business Insights Consulting Analyst supporting consultancy projects related to data and insights for global clients. Collaborating with clients to enhance strategic decision - making processes in travel management.
Analista Funcional de Banca en DXC Technology, especializado en entornos host y programación COBOL. Colaboración con equipos técnicos y gestión de requisitos funcionales en banca.
Senior Analyst at SP Energy Networks supporting the expansion of Flexibility Services into operational businesses. Engaging with market participants and driving network operability.
Junior Automation Developer improving cybersecurity operations tools at Telefónica Tech. Engaging in code and platform modifications, working with security solutions and automation processes.