The inability for offline play bothers me the most, even though it's unlikely to affect me, and to a lesser extent, I have concerns over losing the ability to play these games at the console's end-of-life point. I can trust Valve enough when they say they'll find a way to let me keep any games I've purchased from them should Steam die, but I'm not certain I'll be comfortable with any reassurance Microsoft provides.
There's still a chance I buy the Xbox One, since its TV stuff intrigues me on a tech geek level and Forza Motorsport 5. That's why I'm hoping there's a Forza demo at PAX Prime in August, so I can see if the new edition and what it offers is intriguing enough. If not, or if there's enough missteps, I might end up going PC-only.