Machine Learning Engineer engaged in recommendation and search, NLP, and image processing. Contributing to a major online dating app, Pairs, in Japan.
D-
Hybrid Machine Learning Intern – Dynamic KV-Cache Modeling for Efficient LLM Inference
Posted 2 months ago
About the role
- Machine Learning Intern at D-Matrix developing dynamic KV-Cache for efficient LLM inference, working with advanced memory techniques.
Responsibilities
- Research and analyze existing KV-Cache implementations used in LLM inference, particularly those utilizing lists of past-key-values PyTorch tensors.
- Investigate “Paged Attention” mechanisms that leverage dedicated CUDA data structures to optimize memory for variable sequence lengths.
- Design and implement a torch-native dynamic KV-Cache model that can be integrated seamlessly within PyTorch.
- Model KV-Cache behavior within the PyTorch compute graph to improve compatibility with torch.compile and facilitate the export of the compute graph.
- Conduct experiments to evaluate memory utilization and inference efficiency on D-Matrix hardware.
Requirements
- Currently pursuing a degree in Computer Science, Electrical Engineering, Machine Learning, or a related field.
- Familiarity with PyTorch and deep learning concepts, particularly regarding model optimization and memory management.
- Understanding of CUDA programming and hardware-accelerated computation (experience with CUDA is a plus).
- Strong programming skills in Python, with experience in PyTorch.
- Analytical mindset with the ability to approach problems creatively.
Benefits
- Medical/Dental/Vision/401k
- Inclusive rewards plan
- Professional development opportunities
Job title
Job type
Experience level
Salary
Degree requirement
Tech skills
Location requirements
Senior Scientist II leading innovative AI and machine learning projects in oncology at Tempus. Collaborating with teams to advance predictive modeling and drug R&D initiatives.
Technical leader architecting and deploying advanced Gen AI and ML solutions for Walmart Connect. Driving innovation in ad targeting, personalization, and campaign analytics.
MLOps Engineer scaling AI/ML solutions across game studios at Stillfront. Collaborating with teams to operationalize machine learning solutions for a diverse gaming portfolio.
Machine Learning Engineer supporting the Enterprise Machine Learning team in developing advanced solutions. Collaborating with stakeholders and driving data science initiatives in a hybrid work environment.
Contract AI/ML Engineer focused on delivering machine learning projects for clients at AND Digital. Aiming to enhance AI - powered solutions and contribute to digital skills development.
Senior Machine Learning Engineer at TomTom building data pipelines for autonomous vehicle mapping solutions. Collaborating in a diverse team to innovate and implement machine learning technologies.
Machine Learning Manager improving automated decision - making and managing a team. Driving innovations in credit risk modeling for Monzo’s borrowing products.
Machine Learning Engineer responsible for ML model design and deployment at SiGMA Group. Enhancing event experiences through AI - driven solutions in iGaming and tech.
Machine Learning Intern at Nomagic tackling physical manipulation challenges in AI robotics with top professionals. Engage in innovative projects while shaping robotic technology.