Int4 and Int8 have certainly a lot of applications in the machine learning field. But I doubt their usefulness for image reconstruction a la DLSS. A similar technique like Facebooks Neural Super Sampling also uses 16 bits of precision.
And people really underestimate how much arithmetic power these neural networks need at runtime. Facebooks algorithm ran on a Titan V and took 18ms to upscale 540p to 1080p
If thatâs the case, unless they plan to utilize the FP16 support, it doesnât sound like they have a solution for ML image reconstruction.
I think people need to accept AMD and therefore consoles wonât have an equivalent to DLSS. Seems like the best gaming purchase this year will be a new graphics card. (From Nvidia)
They have FidelityFX and Resharpener which does the same afaik
It will take serious r&d to make dlss like stuff possible on next gen consoles. Iâm not even sure Microsoft Research is working on it, the last time they demonstrated Super Resolution with Forza Horizon 3 the DirectX team used models from NVidia.
Thats not the same. AMD just sharpens the image to combat the softness of TAA. And you can see the halos from the sharpening filter in some of their demo material.
NVidia invested lots of time and money into machine learning hardware and software. AMD did not. So naturally there is a big gap. And i expect it to grow when we see Ampere and the RDNA2 cards. DLSS is also not open source and can not be easy replicated, so referencing it in context of next gen consoles from AMD is a bit pointless.
At least compared the DLSS 1.0 the combination of the AMD methods resulted in better IQ than what DLSS was capable of and with less resource impact when it comes to frame time budget. May have changed with DLSS 2.0. For console the AMD solution with the less required resources is an adequate solution imo.
Interesting video on GI probes in Forza incase anyone is interested.
If you mean the patent that was touted as related to PS5 DLSS, thatâs a patent for a camera. Has nothing to do with PS5 the console. PS5 itself has no ML hardware of any kind at all as per Sonyâs lead PS5 graphics engineer.
They have texture upscale models of their own that their XGS studios are working with. Already confirmed. Also, DirectML is highlighted as a core tech feature in XSX for a reason. âGeatly improving image qualityâ is specifically what they note wrt their ML hardware here.
DirectML is just a software API for hardware abstraction. This says nothing about the software or inference models you can run on it.
It would be really good to have a cheap upscaling or denoising solution. Its needed for raytracing.
Yes I know that, but they are not going to directly say it can be utilized specifically on XSX to significantly improve image quality if that is not the case. And we know for certain that some studios are using texture upscaling at runtime (i.e. using the ML hardware), so not sure how much more concrete you can get here.
DirectML supports using FP16. This was from 2 years ago.
And heres a code sample:
Awesome post! =D
Looks like the sample from Siggraph 2018
The upscaling model is the one from NVidia. I canât run this right now on my laptop. Has somebody tried?
I think youâll need the latest Win10 insider build plus SDK plus beta driver to build this.
One thing I want to know is what will performance be like when using an external sata SSD usb drive, xsx exclusives wont run off it but it would be great if it still can be used to transfer games @ 500GB/s
This was put together very nicely. The little drop down menus and all make this pretty sophisticated on mobile. Getting used to it all.
Canât wait for any info coming out of hotchips later today.
I am also very curious if we will learn something new.
Yeah, I was wondering the same, I guess it will be dictated by if XSX supports UASP or not. If not we are stuck at BOT speeds <250Mb/s
Love your brain OP. Amazing post and thread 