He is now in my party and, after doing some research, it looks to be damn near impossible to get rid of him. Should I start a new game or is it possible to live with the adverse effects he brings to my party?

I say load a old save if have some. Reminds me of meeting those friggin hubologist and permanently taking away 1luck. Man that pissed me off.

Kill him in the de-brainier.

I've never encountered the Pariah Dog but if shooting him doesn't work then you could try using a lot of super stimpacks on him since they drain health after a while.
Edit:  I just checked the Fallout wiki and the super stimpack trick won't work unless you have a shitload of them since the dog has 750 HP and a super stimpack drains 9 HP