I had the exact same issue. After beating the second dungeon or whatever it told me to fast travel somewhere but it then wouldnt let me as long as I was inside. Once I got outside the dungeon it worked fine.
Yeah it's pretty great. They have an assist system set up where at the most basic level you are only steering, the game handles the gas and brake. You can go from only steering all the way down the crazy sim racing hole, I love Forza 4. Shame the new one looks more arcade-y