Let's just excuse the plot holes that have nothing to do with time travel and let's not even talk about those that do involve it. I enjoyed the movie enough, but I don't see why it's hard to write about time travel in such a way that doesn't get broken to pieces by the end of the movie. I've seen Back to the Future, but I don't remember it well enough to know if it worked or not. I've seen several anime movies/shows (The Girl Who Leapt Through Time, Madoka and Stein's; Gate) where it seems to work as nothing really jumped out at me as being broken when I saw them.
When you have so many different rule sets to choose from why is it so hard?