I have some category ideas for Undertale, but don't know how to submit them.

A mod has to create them. Talk to the mods (and for a game as popular as Undertale, talking to the community as a whole is a good idea). I doubt they'll implement your ideas but it's not IMpossible.

Yeah, mods of the game will probably not add your new category. However, mods of Undertale Category Extensions might -- you should post your ideas in their forum (and see if it already exists)! https://www.speedrun.com/undertalecategoryextensions

Many popular speedrun games have "Category Extension" pages for new additions like yours. Type it in the search bar and you'll see what I mean!