So, I got a bunch of mods (5 different ones, some of them with 5-6 files each) and now the game crashes. Not really a big surprise.

However, I have spent the last 45 minutes turning on and off random mods, starting new games, and still having the game crash on me.

Is there some fancy trick to figuring out where the conflicts are? Or do I just have hours and hours of turning things on and off to go?

When everything else fails re-install the game.

" When everything else fails re-install the game. "

Well dude you can load up that save once you reinstall the game provided you remove any mod items you might have equipped at the time.
I assume you downloaded the mods from fallout nexus?
Go there find the mods you have installed and check for the compatibility issues/bugs.
Like i said re-installing the game should fix it,but that can be considered a last resort.