Why, does it require Lovelace?
Yes. More specifically, it's dependent on Lovelace's Optical Flow Accelerator (OFA), which tracks changes between frames to feed into the algorithm. As far as I can tell the rest of the hardware requirements would be the same as DLSS 2.
Edit: As pointed out by @ILikeFeet below, other Nvidia GPUs have actually had OFAs for a while, so this isn't specific for Orin or Drake. Ignore the rest of this post!
However, there's another Nvidia chip already shipping with an OFA: Orin. It's a useful tool in computer vision, so it's relatively unsurprising to see it in a chip designed for automotive and robotic use-cases.
Here's the thing, though: I think Drake has an Optical Flow Accelerator too. Taking a look through some of the L4T source code posted above, let's take a look at
hardware_t23x.h and
hardware_t239.h, which are header files for the T234 and T239 respectively, containing register definitions for hardware blocks the video driver interacts with. Before diving into them, I should note that they're over a year old now. The T239 file is from last summer, and
was removed from the public repo shortly after.
Here's a block from the T239 header file:
Code:
/* sync registers */
#define NV_HOST1X_SYNCPT_NB_PTS 1024
#define NV_HOST1X_NB_MLOCKS 24
#define NV_HOST1X_MLOCK_ID_NVCSI 9
#define NV_HOST1X_MLOCK_ID_ISP 10
#define NV_HOST1X_MLOCK_ID_ISP_THI 11
#define NV_HOST1X_MLOCK_ID_VI 17
#define NV_HOST1X_MLOCK_ID_VI_THI 7
#define NV_HOST1X_MLOCK_ID_VI2 13
#define NV_HOST1X_MLOCK_ID_VI2_THI 14
#define NV_HOST1X_MLOCK_ID_VIC 18
#define NV_HOST1X_MLOCK_ID_NVENC 19
#define NV_HOST1X_MLOCK_ID_NVDEC 20
#define NV_HOST1X_MLOCK_ID_NVJPG 21
#define NV_HOST1X_MLOCK_ID_NVJPG1 16
#define NV_HOST1X_MLOCK_ID_TSEC 22
#define NV_HOST1X_MLOCK_ID_OFA 8
Here's the equivalent block from the T239 header file:
Code:
/* sync registers */
#define NV_HOST1X_SYNCPT_NB_PTS 512
#define NV_HOST1X_NB_MLOCKS 14
#define NV_HOST1X_MLOCK_ID_VIC 8
#define NV_HOST1X_MLOCK_ID_NVENC 9
#define NV_HOST1X_MLOCK_ID_NVDEC 10
#define NV_HOST1X_MLOCK_ID_TSEC 11
#define NV_HOST1X_MLOCK_ID_FDE 12
#define NV_HOST1X_MLOCK_ID_OFA 13
Unsurprisingly, the T239 code is defining far fewer blocks here, as a lot of the automotive-focussed hardware has been removed. We still have NVENC and NVDEC, the Nvidia video encoder and decoder blocks. We also have VIC, which is the video image compositor, which performs various 2D image processing and compositing tasks. There's the TSEC, which I assume is the Tegra security co-processor, and the FDE, which from the acronym I assume is a full-disk encryption block.
Then, at the end, the OFA, or the Optical Flow Accelerator. Still there on T239, even with other components removed.
There are a whole host of reasons this doesn't mean DLSS 3 support, though. Firstly, this is year-old pre-silicon code. Secondly, we have no reason to believe that the OFA in Orin is even comparable to the one in Lovelace, it may not have either the performance or functionality to produce the inputs required for DLSS 3 frame generation. Finally, even if Drake had a compatible OFA, I'm not sure if DLSS 3 would even be feasible on the chip.
More specifically, it seems to me that DLSS 3 is likely to be quite memory and bandwidth intensive. Assuming you're scaling from 1080p to 4K, you have to keep two full 4K frames in memory, feed them into the OFA, write the result back out to memory, and then feed all three of those into the frame generation part of the algorithm, along with motion vectors, depth buffers, etc. Then another 4K frame gets written back into memory. For an RTX 4090 with 1TB/s of bandwidth moving all this data back and forth probably isn't that big of a deal, but for Drake with ~100GB/s of bandwidth it may not even be feasible at all.
Bonus content: It looks like T239 might support SD Express cards, if
this commit message is anything to go by. Of course the same seems to be true for both T194 (Xavier) and T234, so it likely has no bearing on Nintendo's decision on what kind of removable storage to use in the new model, but I thought it was interesting nonetheless.
