Nvidia NIM Platform Will Accelerate AI Model Deployment

At the recently held annual technology show GTC 2024, Nvidia announced the launch of its latest software platform called NIM. The main objective of the new product is to speed up the process of introducing artificial intelligence models. As a result, developers will be able to deploy them more efficiently.

According to Nvidia representatives, the creation of such platforms usually takes a long time – from several weeks to months. And this is with a team of highly qualified AI specialists. The use of NIM involves the formation of an infrastructure from ready-to-use containers with AI based on Nvidia equipment. As such, the platform provides a comprehensive software foundation for organizations looking to accelerate their AI missions.

NIM currently supports models directly from NVIDIA, as well as from companies such as A121, Adept, Cohere, Getty Images, and Shutterstock. In addition to them, it supports open models from Google, Hugging Face, Meta, Microsoft, Mistral AI, and Stability AI. Nvidia is actively partnering with leading technology companies Amazon, Google, and Microsoft to bring its NIM microservices to SageMaker, Kubernetes Engine, and Azure AI platforms. In the future, they are planned to integrate with Deepset, LangChain, and LlamaIndex.

Manuvir Das, director of enterprise computing at Nvidia, noted that their GPUs are an ideal place for AI models to run. With NIM, developers have the best software environment for creating enterprise applications. He also emphasized that Nvidia takes on the technical aspects of the work, allowing the authors of artificial intelligence models to focus on what is most important.

To speed up development, Nvidia uses Triton, TensorRT, and TensorRT-LLM servers. The following Nvidia microservices are available through NIM: Riva (for adapting speech models), cuOpt (for optimizing routes), and Earth-2 (for weather forecasting). In the future, the corporation plans to expand the existing functionality.