I noticed that basically every WR of Mini Game Mode has the player only playing 1 mini game, and there's was a user testing to see what game was the fastest. If there isn't a disable button, does this mean to get the fastest time you need to have a save file with a specific game unlocked and ONLY that game?

There are numerous categories within mini-game mode, which one are you referring to? I'm afraid that there is no way to "disable" a mini-game after unlocking it. You will have to do a new save file with only this mini-game unlocked. Are you playing on console or emulator?

I'm referring to every difficulty category on these modes; Wars, Tournament, and Battle. I'm playing on Console.

Well, if you have a modded Nintendo Wii and an SD card/ USB, I can suggest a method using GCMM (https://github.com/suloku/gcmm). Otherwise, you're left to unlock this one mini-game yourself by playing Party Mode on one save file, which can be tedious. If you decide on the latter course, Toy Dream has 9 blue spaces and 1 happening in the first 10 spaces, so high chance of a 4 player mini-game. If you play a 4 player mini-game other than the desired mini-game, you have to reset.