Softness isn’t just based on the oils a fabric does or doesn’t have, but also on the length and flexibility of the fibers the threads of the cloth are made of. If you splurge on quality towels, like supima cotton for instance, no matter how many times you wash your towels (without softener) they will remain soft and still absorbent.