Nvidia VSR Enhancement, totally fake and exaggerated!

Nvidia VSR Testing: AI Upscaling and Enhancement for Video

Posted on

Nvidia Video Super Resolution — Nvidia VSR — officially becomes available to the public today. First previewed at CES 2023, and not to be confused with AMD’s VSR (Virtual Super Resolution), Nvidia VSR aims to do for video what its DLSS technology does for games. Well, sort of. You’ll need one of Nvidia’s best graphics cards for starters, meaning an RTX 30- or 40-series GPU. Of course, you’ll also want to set your expectations appropriately — the above lead image, for example, is faked and exaggerated and not at all representative of VSR.

By now, everyone should be getting quite familiar with some of what deep learning and AI models can accomplish. Whether it’s text-to-image art generation with Stable Diffusion and the like, ChatGPT answering questions and writing articles, self-driving cars, or any number of other possibilities, AI is becoming part of our everyday lives.

The basic summary of the algorithm should sound familiar to anyone with knowledge of DLSS. Take a bunch of paired images, with each pair containing a low-resolution and lower bitrate version of a higher resolution (and higher quality) video frame, and run that through a deep learning training algorithm to teach the network how to ideally upscale and enhance lower quality input frames into better-looking outputs. There are plenty of differences between VSR and DLSS, of course.

For one, DLSS gets data directly from the game engine, including the current frame, motion vectors, and depth buffers. Combined with the previous frame(s) and the trained AI network to generate upscaled and anti-aliased frames. With VSR, there’s no pre-computed depth buffer or motion vectors to speak of, so everything needs to be done based purely on the video frames. So while in theory VSR could use the current and previous frame data, it appears Nvidia has opted for a pure spatial upscaling approach. But whatever the exact details, let’s talk about how it looks.

(Image credit: Nvidia)

Nvidia provided a sample video showing the before and after output from VSR. If you want the originals, here’s the 1080p upscaled via bilinear sampling source and the 4K VSR upscaled version — hosted on a personal Drive account, so we’ll see how that goes. (Send me an email if you can’t download the videos due to exceeding the bandwidth cap.)

We’re going to skirt potential copyright issues and not include a bunch of our own videos, though we did grab some screenshots of the resulting output from a couple of sports broadcasts to show how it works on other content. What we can say is that slow-moving videos (like Nvidia’s samples) provide the best results, while faster-paced stuff like sports is more difficult, as the frame-to-frame changes can be quite significant. But in general, VSR works pretty well. Here’s a gallery of some comparison screen captures (captured via Nvidia ShadowPlay).

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *