Staff AI Scientist specializing in machine learning fundamentals at HackerRank. Leading rigorous AI evaluation and dataset construction efforts across teams in a hybrid setting.
Responsibilities
Design, prepare, and curate high-quality evaluation datasets with defensible methodology.
Define criteria for dataset construction, ensuring statistical rigor, reproducibility, and fairness.
Develop new metrics and evaluation frameworks to measure model performance in nuanced ways.
Evaluate LLMs and other pre-trained models using carefully chosen datasets and metrics.
Build scalable pipelines for training, fine-tuning, and benchmarking models.
Contribute to projects involving fine-tuning, retrieval-augmented generation (RAG), and other adaptation methods.
Partner with product and engineering to align scientific rigor with business outcomes.
Define evaluation standards and ML lifecycle practices that raise the bar across the company.
Mentor scientists and engineers, guiding best practices in experimentation, statistics, and ML development.
Requirements
Master’s degree (PhD preferred) in Computer Science, Statistics, Machine Learning, or a related quantitative field.
Strong background in mathematical and statistical foundations of machine learning (probability, linear algebra, optimization, experimental design).
Demonstrated experience in end-to-end ML lifecycle: dataset preparation, model training, evaluation, deployment, and monitoring.
Proven expertise in evaluation dataset design and metric creation, not just using existing benchmarks but knowing when and how to improve them.
Experience with LLM evaluation, fine-tuning, and RAG, with the engineering skills to build production-ready pipelines.
Track record of strategic impact at a staff or principal level setting evaluation and research standards across teams.
AI Research Intern focusing on advancing deep learning techniques in financial products at TD. Collaborating on large - scale datasets and representing the team at ML conferences.
AI Research Engineer designing downstream AI models operationalizing clinical endpoints in breast cancer care. Collaborating with clinical experts to enhance healthcare innovation in medical imaging.
Machine Learning Engineer designing, implementing, and deploying ML solutions for GEICO. Collaborating with cross - functional teams to integrate ML models and ensure business impact.
Intern AI Researcher at Analog Devices exploring AI models for efficient edge computing. Collaborating with experts to drive breakthroughs in model optimization and compression.
Senior AI Researcher at Dolby developing audio and video technologies with a focus on deep learning. Partnering with experts to innovate in multimedia analysis, processing, and rendering.
AI Research Lead guiding development and evaluation of African - first open language models at GSMA. Collaborating with researchers and operators to ensure high - quality outcomes across diverse linguistic structures.
AI Scientist Intern focusing on automated speech processing for educational applications at Pearson. Involves mentoring, training ML models, and contributing to prototype ideas.
Lead AI initiatives in multimodal ML/AI at Eluvio AI Labs. Driving innovations for video understanding, content processing, and more in a hybrid environment.
Senior AI Scientist enhancing video and multimodal AI models for Eluvio AI Labs. Developing state - of - the - art models and impacting decentralized content AI and monetization.
AI / Machine Learning Researcher joining Planner 5D's AI team for applied research in home design. Collaborating on real - world product challenges with deep learning methods and data analysis.