Cloth soaks a lot of water up, which makes it take longer to dry. Skin doesn't soak nearly as much up, and dries a lot faster. It's really much better to swim naked, but until then this kinda material is the next best thing. It dries super fast!

*That's horrible for your clothes Sora, you really shouldn't do that. Just because they were too lazy to render you in a swimsuit...*