Driveclub's low scores by and large came from people who don't really like racing games. Jeff's in particular sticks out as one of the most unhelpful to driving game enthusiasts. At some points he just gets stuff flat out wrong (in the review I believe he says you can't change your car's paint color, but you absolutely can). Also aside from Jeff, I don't think any other review took issue with the handling, and from nearly everyone else I've heard from that has played it, the driving is supposed to be best in class.
Poor reviews from people with warped expectations coupled with Driveclub's horrendous server issues seems to have really hurt what could have been a cool, community-driven series.
Fortunately it seems like with the latest weather update people have been giving it a second chance, and I'm once again hearing a lot of positive things, so I guess it's not over quite yet.
You're right. You can paint the car. 10/10.