So what exactly can tensor cores do for AI applications that wouldn't be doable (or easily doable) otherwise? More and more I feel like Nintendo is going to experiment with AI stuff, what type of features and applications could we reasonably expect? Do we think it'll heavily tie into AR/cameras?
This is something that I can finally contribute to being a data scientist.
So in theory, anything you can compute with tensor cores you can still compute with a traditional CUDA core or even on the CPU if you program it in that way, the difference comes from how fast you can do it.
What tensor cores do is allow for really, really fast inference so that images can be reconstructed in real time. So essentially, tensor cores could be used for performing any real time inference tasks where pre trained neural networks are involved.
A few Nintendo like examples.
1. Real time, accurate voice recognition. This could be used for directing squads of AI team mates in games, or even warioware / Mario party style mini games that use sound but more advanced.
2. Real time 3d object detection. So with the use of a camera, more advanced object detection could be done and faster. Again I could see this being used for mini games or interesting gameplay mechanics like enhanced hand gestures. Might even be able to use this to make the motion controls more accurate by tracing the position of the controller in the players hand.
3. Again, with cameras in mind, that goofy face swap stuff could be done in real time too potentially. Why not play Mario Kart 9 with your own face super imposed over Toads for a truly nightmarish experience.
Just a few examples off the top of my head, worth mentioning that DLSS probably won't be usable whilst tensor cores are being used for other inference operations so there would be a sacrifice there.
I'm in the last year of my data science apprenticeship and my field isn't games so someone more knowledgeable and who works in the industry like Brainchild will probably give you better examples.