A couple things.
The crusader is tanky as shit at low levels. Until you hit about lvl 50+ when the elite mobs start getting 3 and 4 traits its super easy. The game scales to your level, and any zone you go into will have mobs of an appropriate level for you so out-leveling your game isn't possible, but out gearing your difficulty certainly is.
Lastly, you're approaching the game in a slightly backwards mindset. This game really starts to ramp up at the level "cap". Keep in mind that though your character has a level cap, there are virtually endless "paragon" levels to continue to gain. Couple that with the classic Diablo loot treadmill and you have pretty much endless content at the endgame. I would go so far as to say you're actively hurting your experience by worrying about leveling too fast. Bump that difficulty up as high as you like and enjoy the challenge. It helps more than you think because you'll be at, or close to, the level cap by the time you finish the story the first time. I had to go to torment 1 for a large part of my crusader leveling, and then drop it down to master on act 5 because the expansion is a bit harder than the vanilla content, but this is going to depend entirely on your gear at that point.