help please, big internet lag spikes while gaming

Hey all,

i need help figuring this out. Currently paying for high speed internet (15 Mbps up/ 1 Mbps down) via my home ISP (WOW, Columbus, OH) but have been having problems with concurrent connections. I.E. trying to play a game on Xbox Live at the same time someone else is streaming a show via netflix. When nobody else is using the internet, it's pretty good, speedtest.net tells me my downstream speed is between 15-16 Mbps and my upload is usually around 2 Mbps. ping to a server within 50 miles of my house is usually less than 50ms. When someone else is using the internet to stream, i periodically get large lag spikes that without my router's QoS settings in place can sometimes be upwards of 500-600 ms just to ping something like google.com. Surely my connection should be sufficient to support the 3-5 Mbps traffic of streaming an HD video at the same time as the <100 kbps traffic from my game. What gives? I've been able to throttle this down to mostly under 150 ms spikes via my router's QoS but for the connection to jump from 30-40 ms to 150 ms or higher every few seconds, can this be right? Anyone have any experience working for a cable company/ISP hat could help

@oldmanlight: What kind of router are you using? Also, what are you gaming on and are you gaming on WI-FI?

modem is a motorola SB6121, router currently is a netgear n750 which replaced my linksys e1200 when i though that might be the problem., playing on a wired connection that's connected by CAT5e all around, ping to gateway is <1ms.