Should the timer start at pressing "Start Adventure" on Go For It's screen? This is because the only thing between the start screen and adventures is some menuing and a cutscene.

If you skip the tutorial, yes.