You're right. It would be an obnoxious to assume that your life has been all roses, just because you havent dished out anything to the opposite effect..which, after all, would be very personal.
Unless someone blurts out specific information about themselves, I tend not to wonder about that information or try to speculate or second guess. It is a common human failure to take things very much at face value, but perhaps We do it a bit more than most.