My hands aren’t particularly small either and I hate big phones. I like to use my phone with one hand most of the time so I need a small phone to hold it comfortably and to reach around the touch screen.
I’ve noticed even few millimeters of difference in width has great impact on how comfortable the phone is to use. Gesture navigation also helps and I think it was created out of necessity for bigger phones.