StarTopic Future Nintendo Hardware & Technology Speculation & Discussion |ST| (Read the staff posts before commenting!)

[MP!]

So my question is like, what could a theoretical Face Raiders 2 do with tensor cores better than the original on 3DS did? What kind of AI enhancements could be made?

Ah I see...
I'm really no expert there...
But I'd assume being able to assess surroundings on a far more accurate level than before? haha All I can think of is bigger better sort of stuff. I suppose it would be up to the software geniuses at Nintendo haha

Edit: with their talks of players facing each other maybe facial recognition?
even a front facing camera that can determine your emotion might be nifty for gameplay implications.
Labo IR camera can recognize shapes ... maybe they crank that up with hand gestures/signs

 

[mjayer]

Problem with async gaming is that it's next to impossible to port forward unless you have hardware capable of the exact same stuff

Which Nintendo themselves said the eShop library for each account will carry over to future systems

But i think it would be a great bet for the Switch platform as it encourages the purchase of multiple consoles per home. Plus I'm guessing the platform is going to be around for more than another decade.

 

[Crusters]

Problem with async gaming is that it's next to impossible to port forward unless you have hardware capable of the exact same stuff

Which Nintendo themselves said the eShop library for each account will carry over to future systems

I do feel like the switches hybrid nature has a mitigating salve for this issue that while certainly not the ideal, is both more than functional and criminally ignored.

These are portable systems that can connect wirelessly with each other.

Just make it a multiplayer feature with multiple switches. Main player docks the switch, other switches brought by freinds are hand held.

 

[Alovon11]

But i think it would be a great bet for the Switch platform as it encourages the purchase of multiple consoles per home. Plus I'm guessing the platform is going to be around for more than another decade.

But it likely isn't worth it unless they really go for the iPhone release model for Drake and future systems

 

[Thraktor]

I'm actually kind of curious about the memory setup on Drake. With my previous expectations for the new hardware I was pretty happy with the idea of a 128-bit LPDDR5 interface for a maximum of ~100GB/s of bandwidth docked, but with as many as 12 SMs I'm wondering if even that wouldn't be enough. The big GPU L2 cache would definitely help reduce bandwidth requirements, but there's only so much that can do. Perhaps a 192-bit interface for around 150GB/s? That would also make 12GB of RAM more likely, but would require more physical space on the motherboard to accommodate 3 memory modules.

So what exactly can tensor cores do for AI applications that wouldn't be doable (or easily doable) otherwise? More and more I feel like Nintendo is going to experiment with AI stuff, what type of features and applications could we reasonably expect? Do we think it'll heavily tie into AR/cameras?

Yeah, AR is definitely a potential use-case, I'd say machine learning would be well-suited to the general object-tracking and depth-map generation which are standard in AR. I'm most interested in rendering use-cases, though, particularly using a final ML stage in rendering for non-realistic rendering styles. Nintendo more than basically any other developer makes use of non-realistic rendering, and ML could fit into that really well, potentially making games that are indistinguishable from paintings (apart from the fact they're moving!) or producing any other kind of visual style Nintendo's designers are interested in.

Well I doubt they'd pre-binn Drake down from 12 to anything less than 8

If they are binning, then my expectation is that they'd bin one TPC (so two SMs), bringing them down to 10.

 

[ShaunSwitch]

So what exactly can tensor cores do for AI applications that wouldn't be doable (or easily doable) otherwise? More and more I feel like Nintendo is going to experiment with AI stuff, what type of features and applications could we reasonably expect? Do we think it'll heavily tie into AR/cameras?

This is something that I can finally contribute to being a data scientist.

So in theory, anything you can compute with tensor cores you can still compute with a traditional CUDA core or even on the CPU if you program it in that way, the difference comes from how fast you can do it.

What tensor cores do is allow for really, really fast inference so that images can be reconstructed in real time. So essentially, tensor cores could be used for performing any real time inference tasks where pre trained neural networks are involved.

A few Nintendo like examples.

1. Real time, accurate voice recognition. This could be used for directing squads of AI team mates in games, or even warioware / Mario party style mini games that use sound but more advanced.

2. Real time 3d object detection. So with the use of a camera, more advanced object detection could be done and faster. Again I could see this being used for mini games or interesting gameplay mechanics like enhanced hand gestures. Might even be able to use this to make the motion controls more accurate by tracing the position of the controller in the players hand.

3. Again, with cameras in mind, that goofy face swap stuff could be done in real time too potentially. Why not play Mario Kart 9 with your own face super imposed over Toads for a truly nightmarish experience.

Just a few examples off the top of my head, worth mentioning that DLSS probably won't be usable whilst tensor cores are being used for other inference operations so there would be a sacrifice there.

I'm in the last year of my data science apprenticeship and my field isn't games so someone more knowledgeable and who works in the industry like Brainchild will probably give you better examples.

 

[MP!]

How I've always envisioned a VR Switch is just that... it's a VR headset with a dock that allows it to switch to a regular TV console. So.. the same as the current Switch but "handheld mode" now means VR.

Seems like a great idea to me. You'll get people getting their 2nd (or 3rd?) Switch to experience VR and people who don't have one but may be interested for a killer VR app (Metroid? Zelda?) get the value proposition of not just a new VR headset but a new Nintendo Console.

I also wonder if after this Switch 2 or Pro launches Nintendo will start dabbling again with asynchronous gameplay like we got with Nintendoland and some other Wii U games. Would be great for VR as well.

something like this is very nintendo like in my opinion. multiple people play on the TV with controllers... one person on VR ...
Alternatively like you state, multiple switches could be used here as well.

I think it's something Nintendo could do a lot with.... though I'm not fully expecting them to dive in.
Seems like their primary focus is getting people engaged with /signed up with switch online

 

[NineTailSage]

I think the difference between A78 and A78C is basically academic in this case, as even the cluster and L3 cache size limitations are only going to have an impact if you use ARM's core interconnect, cluster and cache implementation, which Nvidia likely aren't using (they didn't on TX2, TX2 or Xavier, and although there isn't any public confirmation on Orin, I'd suspect it's a custom interconnect too). The distinction between A78 and A78AE is even a bit blurry, as Nvidia aren't using the main feature of A78AE (the ability to run pairs of cores in lock-step), and they seem to switch back and forth between calling Orin's CPU cores A78 or A78AE.



I wouldn't say 1.5GHz is that high for RDNA2, as the desktop cards all comfortably boost over 2GHz, and even the PS5 can hit up to 2.2GHz. Although I'm more thinking about binning for physical defects, rather than binning for clocks.

Not entirely sure about the Series S but didn't Microsoft state that they go back and forth on Series X with some APU's having less CU's but higher or lower clocks?

Solving for Yield: The Effect of the GPU​

Console processors are different to desktop and mobile processors in the sense that there is no SoC binning. For any given silicon product that is manufactured, there will be both a variability in transistor performance as well as defined defects in the design. The goal of the manufacturing process is to provide the best of both, naturally! For a given design, consumer processors in PCs and laptops will be put into different ‘bins’ and assigned different names and values based on transistor performance. Console processors by contrast have to all perform the same in order to meet a minimum performance requirement, and there is no binning. A console manufacturer has to use a design and a performance point such that as many processors as possible from the production line meet that point. This is part of the yield equation for any console processor.

We’ve covered above a number of design choices that Microsoft made in this article, some of which factor into that binning equation and making sure the design gets the highest yield possible. One other factor we haven’t specifically touched on yet is the GPU. The Scarlett SoC physically has 56 compute units for graphics, but only uses 52 in the retail product. The presentation at ISSCC spent some time going into the upsides of both options, but ultimately why Microsoft went with 52.

Xbox Series X SoC: Power, Thermal, and Yield Tradeoffs

www.anandtech.com

 

[MP!]

1. Real time, accurate voice recognition. This could be used for directing squads of AI team mates in games, or even warioware / Mario party style mini games that use sound but more advanced.

NINTENDOGS where your dog actually listens to you
The cat's don't though ... to keep it realistic

 

[Crusters]

I'm actually kind of curious about the memory setup on Drake. With my previous expectations for the new hardware I was pretty happy with the idea of a 128-bit LPDDR5 interface for a maximum of ~100GB/s of bandwidth docked, but with as many as 12 SMs I'm wondering if even that wouldn't be enough. The big GPU L2 cache would definitely help reduce bandwidth requirements, but there's only so much that can do. Perhaps a 192-bit interface for around 150GB/s? That would also make 12GB of RAM more likely, but would require more physical space on the motherboard to accommodate 3 memory modules.

I'm feeling pretty adrift on this, but not in a bad way. After a full ampere gpc of 12 SM'S and not something like an offshoot offshoot with 6 or something..... I was expecting like, half the bandwidth of your maximum, so... I'm just looking for things to reassess and gain bearings and direction again.

If they go in hard on the confidence of their bandwidth compression technologies, what do you think the lowest they could go without.... Obviously blowing smoke up even their own keister's could be?

 

[Alovon11]

If they are binning, then my expectation is that they'd bin one TPC (so two SMs), bringing them down to 10.

So still likely in PS4 Pro territory still when docked due to the increased clock that likely would bring

 

[Skittzo]

This is something that I can finally contribute to being a data scientist.

So in theory, anything you can compute with tensor cores you can still compute with a traditional CUDA core or even on the CPU if you program it in that way, the difference comes from how fast you can do it.

What tensor cores do is allow for really, really fast inference so that images can be reconstructed in real time. So essentially, tensor cores could be used for performing any real time inference tasks where pre trained neural networks are involved.

A few Nintendo like examples.

1. Real time, accurate voice recognition. This could be used for directing squads of AI team mates in games, or even warioware / Mario party style mini games that use sound but more advanced.

2. Real time 3d object detection. So with the use of a camera, more advanced object detection could be done and faster. Again I could see this being used for mini games or interesting gameplay mechanics like enhanced hand gestures. Might even be able to use this to make the motion controls more accurate by tracing the position of the controller in the players hand.

3. Again, with cameras in mind, that goofy face swap stuff could be done in real time too potentially. Why not play Mario Kart 9 with your own face super imposed over Toads for a truly nightmarish experience.

Just a few examples off the top of my head, worth mentioning that DLSS probably won't be usable whilst tensor cores are being used for other inference operations so there would be a sacrifice there.

I'm in the last year of my data science apprenticeship and my field isn't games so someone more knowledgeable and who works in the industry like Brainchild will probably give you better examples.

Thank you for the detailed response.

This part-

2. Real time 3d object detection. So with the use of a camera, more advanced object detection could be done and faster. Again I could see this being used for mini games or interesting gameplay mechanics like enhanced hand gestures. Might even be able to use this to make the motion controls more accurate by tracing the position of the controller in the players hand.

Is quite interesting because I actually had a sort of similar idea back when the NX was still the NX, for a way to replace Wii style pointer controls with eye tracking technology. This is something tensor cores could theoretically help with, yes? Tracking the position/aim of your eyes and then putting a reticle on the screen at that point.

 

[PedroNavajas]

If they are binning, then my expectation is that they'd bin one TPC (so two SMs), bringing them down to 10.

They are binning from 16 to 12! (Only half joking).

 

[Anatole]

* Hidden text: cannot be quoted. *

This is really fascinating and led me to do a bit of reading, so just to confirm/modify one point:

 

[Mr.Gamerson]

The extra horsepower will come in handy if the tensor cores have to be used to do other cool stuff instead of DLSS.

Nintendo can have Switch 4K in 2022/23 then Switch 8K in 2026/2027 and then a complete brand refresh in 2029/2030.

I really like the name Switch 4K but I doubt Nintendo would call it that since they went out of their way to denounce a switch 4k was in development.

 

[Deleted member 887]

So what exactly can tensor cores do for AI applications that wouldn't be doable (or easily doable) otherwise? More and more I feel like Nintendo is going to experiment with AI stuff, what type of features and applications could we reasonably expect? Do we think it'll heavily tie into AR/cameras?

I'm in the last year of my data science apprenticeship and my field isn't games so someone more knowledgeable and who works in the industry like Brainchild will probably give you better examples.

Procedural generation I think is the big one missing from the list. Enemies, maps, NPCs, even text could be generated by AI. You're already seeing that - hell, No Man's Sky is coming to Switch! - but Off The Shelf models that are accelerated by tensor core accelerated could absolutely make AI procedural generation open to devs without an entire engineering team dedicated to performance.

Only potential challenge here is that the performance of your real time AI is going to depend on whether you are docked or undocked. For DLSS that isn't super problematic, as the performance is dropping at the same time your target resolution is. Other AI uses might not have the same synergy.

 

[Mr.Gamerson]

Maybe Nintendo's plan as well is to use NIS or FSR in the event they decide to use the tensor cores for something else.

 

[Hermii]

Maybe Nintendo's plan as well is to use NIS or FSR in the event they decide to use the tensor cores for something else.

At least those are always available.

 

[Skittzo]

Procedural generation I think is the big one missing from the list. Enemies, maps, NPCs, even text could be generated by AI. You're already seeing that - hell, No Man's Sky is coming to Switch! - but Off The Shelf models that are accelerated by tensor core accelerated could absolutely make AI procedural generation open to devs without an entire engineering team dedicated to performance.

Only potential challenge here is that the performance of your real time AI is going to depend on whether you are docked or undocked. For DLSS that isn't super problematic, as the performance is dropping at the same time your target resolution is. Other AI uses might not have the same synergy.

Could deactivating SMs instead of lowering clocks be an answer to that? I guess that would probably present the same problem, TCs that were running tasks would need to hand them off to the active ones thus adding to their load.

 

[Deleted member 2548]

With the impending release of more files in the next ~24 hrs or so, what's the best case scenario for details about the SoC that'll be obtained? Or what kind of blind spots are expected to remain? For example, I'd imagine we'd get the CPU core count, but we still won't know what clocks / profiles Nintendo plans on using.

 

[Pokemaniac]

Sure would be nice if the new dock for the new unit could just use the existing HDMI/USB-C power adapter of the current Switch Dock so I didn’t have to run any new cables in my entertainment centre. Like I hope the HDMI cable of the current Switch is 4K HDR compatible and that the current power cable is good enough

There are several variables that are hard to predict with dock compatibility. I will say it's probably more likely that Drake's dock will be fully BC with existing Switches than the other way around, but there are a bunch of possible outcomes.

One thing that I am pretty confident in is that Drake won't be fully functional in the launch docks, though. You're definitely going to be giving up something if you use it with one of those, if it works at all.

Apologies if this seems like a silly question, since I’m not a techy person:

To allow older switch models to be able to play more intensive games made for the Pro/2, maybe they’d have the portable mode of more intensive games play in docked mode.

This would cause an issue with the Lite, but maybe Nintendo would allow developers to overclock the Lite or something.

Would that be possible?

Battery life would become an issue for the Lite, but at least the option would be there.

They could, in theory, allow this, but probably won't. Switch Lite in particular would require that all the units were validated at the docked clocks for that to be safe, which isn't a guarantee.

 

[Derachi]

There are several variables that are hard to predict with dock compatibility. I will say it's probably more likely that Drake's dock will be fully BC with existing Switches than the other way around, but there are a bunch of possible outcomes.

One thing that I am pretty confident in is that Drake won't be fully functional in the launch docks, though. You're definitely going to be giving up something if you use it with one of those, if it works at all.

I'd have no trouble replacing the dock itself. I just don't want to have to re-run cables. The actual dock on the end of those cables is interchangeable. I guess I'm basically hoping the HDMI cable that came with my launch Switch is 2.0 compliant. 2.1? I don't know which HDMI we're on anymore haha

Also that the launch power supply would continue to be enough.

the docked clocks

I read this as "The Docked Docks"

 

[Hermii]

Since we have a pretty good idea about the hardware already, what more are we expecting to leak tomorrow?

 

[Skittzo]

Since we have a pretty good idea about the hardware already, what more are we expecting to leak tomorrow?

Could be some testing info including discussion of the full chip (i.e. the CPU too).

 

[m051293]

Any details from which timelines could be derived?

 

[Maxrunner]

Could be some testing info including discussion of the full chip (i.e. the CPU too).

what about cpu?
edit: never mind

 

[Alovon11]

Since we have a pretty good idea about the hardware already, what more are we expecting to leak tomorrow?

In regards to Drake itself


CPU Config (Number of Cores, A78 vs A78C vs A78AE)
RAM Config (Amount of RAM, Type of RAM, Bandwidth)
Maximum Clocks the SoC could reach (IE: In an NVIDIA Shield.etc)
Maybe some extra stuff about NVN2 if the API itself is in there (We already have the NVN2 Driver)

 

[Skittzo]

Any details from which timelines could be derived?

Maybe, who knows. I'm guessing if there's anything relating to testing it can give us a decent idea of where it is, if physically exists or is taped out.

what about cpu?

It could give us info about the exact CPU configuration. All we know at the moment is it'll use some sort of A78s. No idea of the number.

 

[Pokemaniac]

I'd have no trouble replacing the dock itself. I just don't want to have to re-run cables. The actual dock on the end of those cables is interchangeable. I guess I'm basically hoping the HDMI cable that came with my launch Switch is 2.0 compliant. 2.1? I don't know which HDMI we're on anymore haha

Also that the launch power supply would continue to be enough.

Often old HDMI cables are good enough to meet the spec for new versions, but I'm not sure how likely that would be if they go for HDMI 2.1.

 

[chocolate_supra]

I read this as "The Docked Docks"

Dire, Dire, Clocks

 

[Skittzo]

Dire, Dire, Clocks

Bowser in the Drake World

 

[Legion of Primes]

Bowser in the Drake World

Nintendo : Drake’s deception

 

[Alovon11]

Nintendo : Drake’s deception

Nintendo: A Switch's End

 

[brainchild]

So what exactly can tensor cores do for AI applications that wouldn't be doable (or easily doable) otherwise? More and more I feel like Nintendo is going to experiment with AI stuff, what type of features and applications could we reasonably expect? Do we think it'll heavily tie into AR/cameras?

The ELI5 is that the tensor operations work really well with the concept of neural networks (AI brain) because of how easily and quickly they can be used to represent and correlate data points in multiple dimensions. And by dimensions, I'm talking about categories of information. Having data correlated in many dimensions with simple groups of numbers allow AI to be used in many different contexts, and using hardware that can take advantage of tensor operations is a very efficient way to do machine learning. It's really the implementation of the math (in terms of machine code efficiency) that's easy and convenient for machine learning. That being said, tensors aren't the only way to do machine learning and data scientists are always looking for more efficient methods to represent and process data.

As for examples in the context of video game hardware (aside from the image quality enhancements), AR (tagging/recognition), real-time gameplay analyses/insights/recommendations, more sophisticated NPC interactions, etc. There's a lot you can do with machine learning using tensor operations.

 

[Dakhil]

Maybe Nintendo's plan as well is to use NIS or FSR in the event they decide to use the tensor cores for something else.

Don't forget the DP4a variant of Intel XeSS since Drake should have DP4a instructions support.

 

[Akoig]

Would more information help predict when it could potentially come out or would that be something too broad to guess? I read something about the documentation being 2019, but I guess that isn't enough for making predictions

 

[SiG]

On an off topic note, if anyone has any thread title change suggestion, please send me a conversation since I won't have much time to check every post here.

Submitted mine!

 

[Skittzo]

Would more information help predict when it could potentially come out or would that be something too broad to guess? I read something about the documentation being 2019, but I guess that isn't enough for making predictions

From what I understand some(/all?) of the files originated in 2019 but some(/many?) were updated as of February 2022. That would suggest it's been in the works since 2019 and is still currently in the works.

 

[JoshuaJSlone]

There are third party headsets that exit for switch already for VR purposes, so hopefully Nintendo would find a way to make one that is comfortable for most. Metroid Prime Remake with VR support would be a great way to sell the VR focused sku as well.

Yeah, I've tried such things, and my reaction is "Wow, that is WAY the hell more front-heavy than the Go or Quest or Quest 2." The most comfortable position was lying on my back looking straight up. Unless they make a Switch where not only the controllers separate, but so can the screen, I don't see head-mounting it being a very good solution.

 

[ITSMILNER]

Does anyone think Nintendo could lock some kind of feature/New system behind this new Switch? Either console based voice chat or perhaps GameCube games on the service similar to how they locked some systems behind New 3DS

 

[BlueKing]

Does anyone think Nintendo could lock some kind of feature/New system behind this new Switch? Either console based voice chat or perhaps GameCube games on the service similar to how they locked some systems behind New 3DS

I don't think Nintendo wants any kind of voice chat on their consoles, especially with all the horror stories about grooming and the like.

 

[Skittzo]

Does anyone think Nintendo could lock some kind of feature/New system behind this new Switch? Either console based voice chat or perhaps GameCube games on the service similar to how they locked some systems behind New 3DS

Probably not voice chat but yeah I can certainly see some other stuff. Especially AI stuff. Maybe an AI assistant.

Named Aisha or Aiden. Not ADAM.

 

[Serif]

A long time ago I thought they would lock Bluetooth audio to the successor, but since they enabled it in a software update Ive reconsidered. Considering the huge install base, I don't think they want to lock anything to Drake that would still be possible on the current Switch.

 

[BlueManifest]

Not sure if this has been discussed before. The OLED model's dock is capable to supplying more power to the Switch console than the previous dock. As you can see in the photo below, the old dock takes in 39w and passes 18w to the Switch; the remaining 21w is reserved for the USB ports. The new dock, however, is capable to passing the full 39w to the console—USB be damned.

Since the OLED model doesn't require more power than the OG or red box Switch, this seems to be a future proofing measure. It might indicate that Nintendo is considering a higher power envelope for the next Switch model, and/or a fast charge feature. 12 SMs? High energy density battery? Sure, why not.

If the new switch did use all 39 watts what would that translate the clock speed to from zombies clock speed list

 

[Skittzo]

If the new switch did use all 39 watts what would that translate the clock speed to from zombies clock speed list

I don't think we can even estimate that without knowing about the CPU, RAM, and storage.

 

[ShadowFox08]

Nintendo: A Hacker's End

Fixed that for you.

I also really like Drake's Deception.

Maybe, "Drake's Deception. A Hacker's End"

@Dakhil


Man 12 SMs sounds too good to be true for Switch 2, but when even Thraktor is hyped about this and thinks it's pausigos plus a 192 bit bus bandwidth, and when people are more optimistic to a node more efficient and newer than 8nm samsung, it's pretty crazy. Usually we are on WUST hype since Wii era days and end up with a "Please understand" like half the clockspeedd we hoped from leakers (I remember when Emily said switch tx1 would be close to xbone, but got the fp16 1 TFLOP count).

We'll see how it plays out and if hackers really do release clock speeds tomorrow. Gonna expect steam deck like specs with 8SMs for docked before DLSS is counted just to not get myself dissapointed. I hope I'm wrong though.

 

[BlueManifest]

I could see them releasing smash bros ultimate ultra for this, all dlc included plus a few new chars, stages and 4k 120 fps

 

[Akoig]

From what I understand some(/all?) of the files originated in 2019 but some(/many?) were updated as of February 2022. That would suggest it's been in the works since 2019 and is still currently in the works.

Makes sense to me. I guess there is no way of getting an aproximate date just from the Nvidia side of things

Not sure if this has been discussed before. The OLED model's dock is capable to supplying more power to the Switch console than the previous dock. As you can see in the photo below, the old dock takes in 39w and passes 18w to the Switch; the remaining 21w is reserved for the USB ports. The new dock, however, is capable to passing the full 39w to the console—USB be damned.

Since the OLED model doesn't require more power than the OG or red box Switch, this seems to be a future proofing measure. It might indicate that Nintendo is considering a higher power envelope for the next Switch model, and/or a fast charge feature. 12 SMs? High energy density battery? Sure, why not.

So potential possibility of using the Drake dock for a OG or OLED Switch tablet?

 

[Dark Cloud]

Since we have a pretty good idea about the hardware already, what more are we expecting to leak tomorrow?

More leaks are coming tomorrow?

 

[Skittzo]

More leaks are coming tomorrow?

The hackers are threatening to release the rest of the stolen info if their demands aren't met by tomorrow (they won't be).