After reading around online, I've finally pieced together  what confused me about the story in the game: the odd way of telling all the parts of the main story.  Mostly the collectible newspapers and the blurry filter they lay over the the WWII flashbacks. 
 
I wasn't sure what the cutscenes you see when you pick up a newspaper had to do with the story until much later.  They seemed to be these little clips of what was looking to be a side story about Fontaine and Courtney, but when they became the main story I was unsure as to why they weren't just played as a regular cutscene.  For me that would have made it feel like a much more full story, and would have taken away some of the confusion as to how everything was coming together. 
  
This also ties into my problems with the flashback sequences.  By the time you get to Vice it was very evident that the flashbacks were extremely important, and not just something to build up a background for Phelps.  But by that point I had lost a lot of details I should have taken from them.  But that underwater looking filter, it made it seem unimportant for some reason, like a little thing you don't need to pay that much attention to.  By the time I realized that characters from the Phelps squad were going to be important players, I had forgotten who was who, all I was really seeing was that Cole was kind of an ass during the war and did things too by-the-books.  It made for a very confusing final couple of hours because I had to play catch-up with all the flashback characters while also trying to discover what was going on with Leland Monroe.  As with the newspapers, if these scenes had been played as a normal cutscene, without a stylized filter.  It would have really filled out the past events that led to what Cole and Kelso discover.  
 
Because of these weird delivery methods, it also makes sense why the affair with Elsa seems so out of the blue.  The reasons for why Cole and Elsa come together makes sense in hindsight, but only after reading the Giant Bomb forums and someone explaining it.  It all basically comes down to one problem I think video games will have as they get closer to being great interactive cinema, especially with the facial tech Team Bondi has created: pacing.  If you want your video game to be like a movie, then make sure the pacing feels like a movie.  I understand video games actually have to have the "game" part in them, but if you're going to write a story like L.A. Noire does, with attempts at a noire style story (considering the ending) and subtext (as in the Elsa and Cole relationship), then make sure the way the plot information is conveyed to the player allows then to discern all of it.   
 
Of course this is all my opinion, but did anyone feel the same way I did.