Senior Infrastructure Software Engineer designing scalable ML inference systems at Baseten. Leading initiatives for high-performance deployment and monitoring of machine learning models.
Responsibilities
Design and architect scalable infrastructure systems for our ML inference platform
Lead optimization of Kubernetes deployments for efficient, cost-effective model serving
Drive enhancements to our inference orchestration layer for complex model deployments
Define monitoring strategies for model performance, latency, and resource utilization
Develop advanced solutions for GPU capacity management and throughput optimization
Establish infrastructure automation standards to streamline ML deployment workflows
Partner with other engineers to translate complex inference requirements into technical solutions
Make critical architectural decisions balancing performance with system reliability
Lead technical discussions and mentor junior engineers on infrastructure best practices
Contribute to long-term technical strategy and infrastructure roadmap
Requirements
Bachelor's degree or higher in Computer Science or related field
5+ years experience building production infrastructure systems
Expert-level proficiency in Go, with Python experience a plus
Deep expertise with Kubernetes in production environments
Extensive experience with major cloud providers (AWS, GCP) and neo-cloud providers (Crusoe, DigitalOcean, Nebius) a plus.
Advanced understanding of distributed systems concepts and performance tuning
Proven experience designing observability systems
Track record of leading technical initiatives and mentoring engineers
Experience with ML/AI workloads and MLOps platforms highly valued
Benefits
Competitive compensation, including meaningful equity.
100% coverage of medical, dental, and vision insurance for employee and dependents
Generous PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
Paid parental leave
Company-facilitated 401(k)
Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
Storage & Backup Management Lead managing SAN/NAS storage and backup platforms at Avenga. Overseeing incident response, collaboration, and compliance with data retention and regulatory requirements.
Senior IAM Engineer at Orro Group managing identity and access management infrastructure for major enterprise clients. Focusing on architecture, implementation, and governance in the cyber security domain.
Junior Linux Rendszergazda maintaining and supporting Linux servers for team.blue Hungary. Actively involved in technical projects and customer support within a hybrid work environment.
Software Developer enhancing Fortinet’s next - gen GenAI platform through software design and development. Involve in software lifecycle from debugging to implementation using cutting - edge LLM technologies.
Integrations Tech Lead at Eeze, focusing on integration strategy and API ecosystem architecture. Leading technical teams and ensuring robust integrations with third - party providers.
Full Stack Developer at Morgan Stanley responsible for developing and enhancing distributed systems. Collaborating with teams to modernize platforms and ensure high availability and security.
Software Developer on Enterprise AI team building solutions that drive JCI's future growth. Collaborating in a fast - paced environment and contributing to innovative solutions.
Senior Software Engineer developing warehouse automation solutions. Collaborating with teams to optimize systems and implement high - performance software.
AI Software Architect developing intelligent driver infotainment systems at Daimler Truck. Collaborating globally to build proof of concept applications and showcase AI possibilities.
Lead Architect Engineer responsible for building end - to - end Data to Decision Systems. Collaboration with multiple engineering teams to develop impactful solutions for Fortune 500 clients.