One thing is sure, only powerful Nintendo home console, would sell much less compared to Switch sales numbers,
power is not what selling Nintendo hardware, its games and features.
Also, miniDVD for GC was never real problem, because GC could use multiply discs if size of game is a problem, thats why GC actually got one best 3rd party support if we count Nintendo platforms, while cartridges were real problem for N64.
Well, it was still slightly propriatary and either you hade more cost for 2 discs and the coresponding alt housing, or you needed to shrink the data somewhat. And having data over 2 discs still needs reorganizing and kopying the data, more testing, etc.
It works, it was not ahuge cost factor, for shure. The GC had other problems, mainly PS2s "mature" image and its DVD capability.
The culture was just at a point where nintendo could not get through, and they WHERE strugling to finish internal projects in time (WW, Sunshine,...) so they kinda rushed them.
Essentially the PS2/XBox generation was the one where we had an increase of presence of western devs in console space, "mature" became generally more accepted (mature media generally), essentially the children and teens that got marketed to with "EXTREEEME" stum, got older.
That was not a time for nintendos strength to shine. After the N64 they needed something that was pure gold to fight Sonys PS in Europe and other parts of the world. The US was the strongest market of N64, but in the EU is was already bombing hard, and GC did not enough more then PS2, but a) was harder to pirate (one aspect that pushed PS1), and could not play dvds (and in europe, i think 80% of dvd players i saw in action where ps2s...)
I really cant see a world where nintendo would not have strugled with the GC, even with a normal DVD drive. Probably less then it has, but still far from a hit.
By 2006 DVDs Players where a cheap comodity, the ones who wanted them had them, and motion controlls where a true novelity...
Heck, thats a year before the IPhone 1 is anounced. What wild times we had back then.