I know this is a big ask, but have you looked into Chromecast integration at all? I've been looking for ways to watch GB videos on my TV (premium ones obviously aren't on YouTube) which don't involve a HTPC or a prior download to the home server.
If it's an option, HTC One or Nexus 5. I have an S3 and Samsung don't make devices which feel nice, IMO.
If it's just between those two phones check them out in person and fiddle around with each for 5 minutes or so. Try putting the Note in your pocket to see whether or not it's too big, things like that. I've ended up falling out of love with my S3 because the damned thing is shaped like a bar of soap and keeps falling out of my hands onto my face when I watch YouTube on it in bed. Those are the kind of things which only first hand experience with the device can get you, and the store clerk will accommodate a couple of demos for a $800-$1000 contract sale.