Once you get the dogs out of the way, Capra becomes a cakewalk. If you need to, you can always go all the way up the steps and then push yourself left onto that ledge and throw stuff down at him.
Later on in the Demon Ruins, Capra becomes a regular enemy. All you need is a bit of patience. Any time he jumps at you or swipes with both swords at once, just move out the way and attack him twice. When he does a single attack, just block and attack once.