How AI Is Fueling NVIDIA GTC

NVIDIA RTX, NVIDIA Omniverse, and NVIDIA AI are the main topics of the conference

NVIDIA GTC (GPU Technology Conference) is a global AI conference for developers that brings together developers, engineers, researchers, inventors, and IT professionals. The conference focuses on AI, computer graphics, data science, machine learning, and autonomous machines.

This year, GTC was held online during September 19–22, 2022. NVIDIA CEO Jensen Huang’s opening statement is exciting: Computing is advancing at incredible speeds. The engine propelling this rocket is accelerated computing, and its fuel is AI.

NVIDIA RTX, NVIDIA Omniverse, and NVIDIA AI are the main topics of the conference. All of them are built on top of GPUs. Let’s take a look at them in detail.

GPUs and CUDA Programming

At GTC 2021, Stephen Jones had a talk, How GPU Computing Works. It deep dives into the architecture of GPU (Graphics processing unit), which is a specialized processor designed to manipulate and alter memory to accelerate the creation of images in a frame buffer, and it is intended to output to a display device. GPUs are used in gaming, workstations, cloud, AI training, autonomous vehicles, etc.

At GTC 2022, Stephen Jones had a follow-up talk, How CUDA Programming Works. CUDA (Compute Unified Device Architecture) is the programming language used in GPUs. The talk explains how hardware design motivates CUDA and how CUDA motivates hardware design.

A GPU is huge — with 221,184 threads, and 17,280 KB of shared memory. To take advantage of a GPU, we should use it efficiently. It is capable of 311,869,440,000,000 operations per second, and a good program keeps the GPU busy with meaningful work through asynchronous execution, resource balancing, and accessing memory linearly.

NVIDIA RTX

NVIDIA RTX (Ray Tracing Texel eXtreme) is the most advanced platform for ray tracing and AI technologies that are revolutionizing the ways we play and create.

What is ray tracing?

Ray tracing is a technique in 3D computer graphics. It models light transport in a wide variety of rendering algorithms to generate digital images. It can simulate a variety of optical effects, such as reflection, refraction, soft shadows, scattering, depth of field, motion blur, caustics, ambient occlusion, and dispersion phenomena.

GPUs and CUDA programming enables real-time ray tracing for graphics cards. Over 150 top games and applications use ray tracing to deliver realistic graphics with incredibly fast performance or cutting-edge new AI features.

RTX is the new standard for performance and efficiency, based on the Ada Lovelace architecture. It aims to design complex large-scale models in architecture and product design, scientific visualization, energy exploration, games, and film and video production. The flagship chip is the RTX 4090 GPU, which is four times faster than its predecessor for ray tracing.

The following screenshot comes from an RTX-generated video. The real-time ray tracing makes it lifelike.

NVIDIA Omniverse

As we mentioned in another article, the metaverse defines 3D spaces that let everyone socialize, learn, collaborate, and play in ways that go beyond the imagination. The metaverse is a collective project of multiple companies. It is created and shared by people all over the world and is open to everyone.

There are three essential elements in metaverse:

• XR (Extended reality) — including AR (Augmented Reality), VR (Virtual Reality), MR (Mixed Reality), and everything in between
• Avatars — online representation of users
• Digital ownership — virtual assets based on blockchain technologies

NVIDIA Omniverse is a platform to build and operate metaverse applications.

It enables lifelike avatars, such as a dragon.

Or a synthetic girl.

Omniverse is a virtual world used to design, build, and operate things in the physical world. In other words, the virtual world is a digital twin of the physical world.

The following screenshot is the digital twin of General Motors’ Michigan Design Studio, where designers, engineers, and marketers can collaborate.

HEAVY.AI is a startup that provides advanced analytics to empower time-sensitive, high-impact decisions with big data. It uses Omniverse to analyze 4G and 5G networks at metro and nationwide scales.

The following screenshot shows the physical world at right, and its digital twin at left.

With XR devices, users can connect to both the physical world and the virtual world in Omniverse. In the following screenshot, additional characters are generated on top of the physical world.

Omniverse includes three elements:

• Computers for users
• Servers to host connections to the Nucleus database and run virtual world simulations with GPUs
• GDN (Graphics Delivery Network) to stream high-performance, low-latency 3D experiences to edge devices

Connections to Omniverse are growing fast. Currently, there are 150 connections. It includes many industries, from retail, transportation, telecommunication, manufacturing, media and entertainment, consumer and luxury goods, to supply chain and logistics. Potentially, it is a $100 trillion business.

NVIDIA Robotics

Robotics, including autonomous vehicles, is the next wave of AI. The breakthrough of deep learning has opened the door to creating systems that can perceive their surroundings, plan a sequence of actions, and perform useful tasks in real-time, every time.

The upcoming NVIDIA Thor is a new generation of robotics processor designed for deep learning. It centralizes numerous computers with enhancement in computer density, energy efficiency, and AI inferencing capabilities. It is used in edge devices.

Currently, an autonomous vehicle uses different computers for parking, active safety, driver monitoring, camera mirrors, cluster, and infotainment. With Thor, these functions can be handled by software on the same computer.

The following paradigm was presented at AI Hardware Summit and Edge AI Summit 2022. It describes AI technologies in the cloud and different layers of edge computing.

This paradigm is consistent with NVIDIA’s concept of two computer systems:

• One computer is in the cloud as the AI factory. It develops AI by processing data, training the AI, simulating the digital twin, and mapping the world.
• One computer is in the edge device to operate the robot by processing the sensors to perceive the environment, stay clear of obstacles, and drive the car to its destination.

NVIDIA DRIVE is an end-to-end platform for autonomous vehicle development and deployment. It includes an AI pipeline that constructs a 3D scene from recorded sensor data. The 3D scene can be augmented with human-created content or AI-generated content. It enables the creation of simulation scenarios on a global scale.

The following is the simulation for the generated snow scenario:

A virtual design studio enables car designers, software engineers, and electronics engineers to collaborate on cars yet to come.

Robotic systems are new types of computers. They are real-time systems that sense the environment, reason about their surroundings and plan actions consistent with their goals.

The fundamental processing pipeline for autonomous vehicles can be applied to all kinds of robotic systems. NVIDIA products have powered many of them in various fields. There are no robots with universal skills, but building robotic systems share common methods and tools.

NVIDIA AI

Today’s AI is built on top of accelerated computing, which is a full-stack challenge, demanding a deep understanding of the problem domain, optimization across every layer of computing, and efficient use of all three chips — CPU, GPU, and DPU.

DPU (Data Processing Unit) is a programmable processor, which has a system on a chip that combines the following components:

• Industry-standard, high-performance, software-programmable multi-core CPU
• High-performance network interface
• Flexible and programmable acceleration engines

NVIDIA continuously provides accelerated computing across the full-stack:

• New chips to boost performance
• New libraries to accelerate critical workloads to science and industry
• New domain frameworks to help develop performant and easily deployable software
• New platforms to deploy software securely, safely, with rich features

It has been announced that there are 3,000 accelerated applications, 12,000 startups, 3.5 million developers, and 35,000 companies running on NVIDIA AI.

CV-CUDA is an open-source library, which is designed to build accelerated end-to-end computer vision and image processing pipelines. Currently, over 80% of internet traffic is video. Special effects that used to require offline processing are now going into the cloud for live video. CV-CUDA can stream while processing, relighting, reposing, blurring backgrounds, super resolution, AI inference, and creating computer graphics for AR.

AI continues to make exponential advances with new algorithms and new frameworks to develop them. A recent breakthrough is the introduction of language processing technologies that enable us to build more intelligent systems with a richer understanding of language than ever before.

Large language models include the following technologies:

• Transformer: It is a neural network that learns context and thus meaning by tracking relationships in sequential data like the words in this sentence.
• BERT (Bidirectional Encoder Representations from Transformers): It is a new method of pre-training language representations that obtains state-of-the-art results on a wide array of natural language processing tasks.
• GPT-3 (Generative Pre-trained Transformer 3): It is an autoregressive language model that produces human-like text. Input a short prompt, and the system generates an entire essay.
• DALL-E 2: It is an AI system that can create realistic images and art from natural language descriptions. The featured image of this article was generated by Anupam Chugh using DALL-E 2.

NVIDIA Hopper transformer engine provides platforms for large language models. It reduces the training time from days to hours, or on larger models, from months to weeks.

NVIDIA NeMo large language model is a prompt learning framework, which adapts pre-trained language models to perform specific tasks by training a companion model with only a few examples. The new models are applied to biology and chemistry for target proteins and drug candidates, and they understand chemicals, proteins, DNA, and RNA sequences.

A recommender is a system that suggests products, services, and information to users based on an analysis of data. It is the engine behind social media, digital advertising, e-commerce, and search.

NVIDIA Grace Hopper is a superchip for giant-scale AI and high-performance computing applications. It delivers up to 10x higher performance for applications running terabytes of data, enabling scientists and researchers to reach unprecedented solutions for the world’s most complex problems.

Grace Hopper is ideally suited for recommender systems. A 120-node Grace Hopper system can process a 70 TB state-of-the-art recommender system.

NVIDIA Partners

NVIDIA has 237 partners in the partner ecosystem. Among them, Domino Data Lab is the leading enterprise MLOps platform. It democratizes GPU access and unleashes AI infrastructure.

Domino Data Lab is a diamond sponsor of GTC. This is a talk about how Domino provides a single pane of glass to productionize the data scientists’ work. It can host deep learning models using GPUs and deploy models at the edge using NVIDIA Fleet Command.

Here is an example of model training data:

Conclusion

We watched NVIDIA GTC 2022 online. AI continues to make exponential advances with new algorithms and new frameworks to develop them. NVIDIA has demonstrated the full product lines to propel AI advancement.

A week before, we attended AI Hardware Summit and Edge AI Summit 2022, sponsored by Synopsys, Atos, Cadence, Graphcore, Qualcomm, Rain, SambaNova, Siemens, AMD, etc. These companies have shown many upcoming new products trying to unseat NVIDIA’s leading position.

It is interesting to see the same AI story being told from two points of view. After all, AI has come a long way in changing the world.

In the previous article, we showed how AI systems detect human features. For comparison, the following screenshot shows how the NVIDIA system detects human features.

AI is a helper, and creativity brings us a better world.

Thanks for reading.

Want to Connect?

If you are interested, check out my directory of web development articles.