Hey @Drew, there's two things that I think will really help you out. First, ya gotta use your aim down sights (RB), especially with the tranq pistol. Second, consider doing more missions at night. The guards have terrible sight at night and you can even use your NVG to gain even more of an upper hand against them. Hope this helps!
(Also on a fun note you should totally research a grenade launcher!)
MrBacca's comments