" First of all, I don't think you could get 4 people in front of Kinect and a TV with enough room to move around in the first place. You would need nearly 20 feet to move your arms around and not be hitting people. Second of all, there is this concept called taking turns. Do 4 people play Wii golf at the same time? "The hero has spoken. All hail.
In all seriousness, this is exactly right. It's much more likely that you would take turns in pairs of 2 if anything, just because that actually makes sense.