Homecoming gets a bad rap, but it's not that bad. In terms of story, it at least tries to tie things back to the cult and the roots of the series and had great boss design.
Downpour is an embarrassment created by people that seemed to actively hate the series the were making a game in.
I agree with both of these assessments. In fact, I actually like Homecoming quite a bit, but yeah, Downpour is absurdly awful.
I haven't gone back to the first four games since each was released, but I do like my memory of them. The Room is odd, but it has a House of Leaves vibe that I can really get behind.