Role Overview:

• Drive design, development, and scaling of Large Language Model (LLM) inference services.
• Build systems for inference serving of popular open source/open weights LLMs and custom models.
• Develop novel techniques for optimizing models and scale platform for millions of users globally.

Responsibilities:

• Design and implement an inference platform optimized for various GPU platforms.
• Manage AI and cloud engineering projects through the entire product development lifecycle.
• Optimize runtime and infrastructure layers for best model performance.
• Build native cross-platform inference support across NVIDIA and AMD GPUs.
• Contribute to open source inference engines to improve performance on DigitalOcean cloud.
• Develop tooling and observability for system health monitoring and auto tuning capabilities.
• Create benchmarking frameworks to test model serving performance.
• Mentor engineers on inference systems, GPU infrastructure, and distributed systems best practices.

Qualifications:

• 2+ years of software engineering experience with interest in distributed systems design, AI/ML, and cloud-scale implementation.
• Deep expertise in cloud computing platforms and modern AI/ML technologies.
• Experience with modern LLMs related to hosting, serving, and optimizing models.
• Bonus: Experience with inference engines like vLLM, SGLang, Modular Max.
• Experience researching, evaluating, and building with open source technologies.
• Proficiency in Python and Go programming languages.
• Ideal: Experience with AMD/NVIDIA GPU platforms and toolsets (CUDA, ROCm).
• Strong ownership mindset with problem-solving drive.
• Appreciation for process and cross-disciplinary collaboration among engineering, operations,

support, and product groups.

• Familiarity with end-to-end quality best practices implementation.
• Experience coordinating across time zones/geographies.
• Experience with infrastructure as code tools like Terraform or Ansible.
• Passion for coaching and mentoring junior engineers.

Work Location:

Hybrid role based in Hyderabad, India.

Additional Information:

The job description also includes company culture highlights such as innovation focus, career development opportunities, employee well-being benefits, reward structure including salary range based on market data plus potential bonuses and equity compensation. DigitalOcean is an equal opportunity employer.