Before you buy a single neutral towel, know the one thing that actually matters: the color on the label (ivory, oatmeal, dove, greige, whatever) will shift depending on your bathroom tile and your light bulbs, so a towel that looked warm and creamy in the store can turn gray and sad under a cool white bulb at home. I learned this the hard way with a stack of "warm white" towels that looked like dishwater the second I got them under my bathroom's LED strip. Check your lighting before you check the swatch.
A well made neutral towel has weight to it, and not in a vague "feels nice" way, I mean you can tell the difference holding two in your hands at the same time. Longer cotton fibers, usually combed or Egyptian or Supima if you want the actual words, give you a towel that doesn't pill or go thin after a year of washing. Look at the border and the hem, a double stitched edge that lies flat means someone thought about it, a single loose stitch that's already fraying in the shop is going to be a hole by month four. The loops should be dense and even, not sparse patches where you can see the woven base through the pile. A disappointing towel announces itself fast, it goes flat, it smells faintly of nothing-quite-right no matter how you wash it, and it never quite dries you off, it just moves the water around.
Where neutral towels fall short is honestly: they are the wrong choice if you actually want your bathroom to say something. A stack of oatmeal towels on an oatmeal wall with a beige bath mat is not calm, it's just quiet in a way that stops being restful and starts being nothing. My mother's whole house is like this and she wonders why nobody wants to linger in her bathroom. Neutral towels work best as a canvas, a soft frame for a bold rug or a wall of color, not as the whole plan. If your bathroom already has nothing happening in it, white or putty towels will not save it, they'll just confirm it.
Day of delivery, unfold the whole towel, not just the corner, because some brands fold in a way that hides thin patches in the middle of the field. Run your hand across the surface for any scratchy spots, that's usually a sign of lower quality cotton or a chemical finish that will wash out in a few cycles anyway. And measure your own towel bar or hook before you order, because a bath sheet that reads generously sized in the photo can hang a full foot longer than your bar, pooling on the floor every single time.