Now that we're a week or so in, I think that I can say that it's not just me being stupid but in fact, a poor UI design of the dash on the Xbox One. I'm not sure why Microsoft tried to shoehorn in a UI that has proven unpopular and unsuccessful on Windows 8, Windows RT and Windows Phone on top of a functional Xbox interface, but boy, they've ruined it. So many things take so much longer to do. I frequently get lost in the interface. They took so many things that they nailed with the 360 and made them substantially worse. I just don't understand it. Some seriously baffling decisions with the party system, friends list and achievements especially. These things are what the dashboard is for! It's what the 360 excelled at! Why do they all launch in separate "apps" now? It's confusing and slow. I guess they expect you to use the Kinect to speed up the process, which would be fine, I guess, if it worked every time. In my experience I either don't know the right command to use or it doesn't recognize me a lot of the time.
This Edge article sums up how I feel pretty well. http://www.edge-online.com/features/xbox-why-the-baffling-incompetence-of-the-xbox-one-interface/
Hopefully it's stuff that they will fix, but seeing as how the 360 seemed to get worse and worse over the last few updates I'm not holding my breath. Also with how Microsoft seems to be "all in" on this interface, well, I'm worried. The PS4 UI isn't perfect but it's much faster and easier to use, in my opinion.