Swimming Toggle has no other effect?

Is there a reason you can not have the swimming skill active after you learn it? I don't see why we have to toggle it on and off when there doesn't appear to be any negative affects to keeping it on all the time.