5 is too many. Plus, you have to have novelty. Sure, it's fun having Rorie or Jason on now, but if they were on every week, it'd quickly be less and less interesting. Save them for every few weeks.
People said the same thing about Drew he started becoming a regular on the podcast and now hes one of the most beloved. Honestly, i find Jason far more interesting then some other members so he has my vote.