When I run with OBS I lose like ~1s per encounter and more with general room loading

As an example I lose 2s before first froggit compared to my pb because of loads

I mean, I don't really think there's anything you can do about it, maybe try lowering your OBS settings or (if applicable) force the game to run on your dedicated GPU if you have a laptop: search Graphics settings, add the game to the list and select "High performance"