The jump from the tower is actually pretty easy and saves close to 8 seconds over hitting the button.

One more major time save I know is possible is using the arch vile in the final room to jump you up the staircase skipping the final button.