I'm a big man, 6' 7" 285lbs and wonder if a Prius would be a reasonable second car for me to use as an around town car. I currently drive a diesel pickup and will be a mile from my job. I really don't want to drive the truck when I can't walk.
Comfort is a really subjective thing.
You are going to have to try different models and decide for YOU.
If you do a lot of other short trips close to home, you might want to consider an all electric model.