Which one should I buy?

Hard one, both have a ton of replay value if you want to collect things. In Riddick you pretty much get two games, but its single player only, where as in Resident Evil 5 you can play co-op.

I should mention that though i had an original Xbox I never played Butcher Bay

Riddick is the better value. I have beat both games (not completely beat Riddick, Dark Athena part is still in progress) but Riddick has more value to it since it does pretty much have 2 games.

 RE5 is ok and does have Co-op. It also took me around 9 or 10 hours to complete. Riddick took me 10 or so hours to complete Butcher Bay.

Try the demos first and see which you feel you like better. Then rent the one you don't like as much.

thanks for the advice guys