It's only natural for leather to soften over time, that's it's nature. the following is a link from a Chuck Burrows post on another site. Personally, I'd do no more than Case the leather. Wet it, put it in a plastic bag and leave it in the refrigerator overnight. When ready the leather should look dry (as before wetting) but feel moist (to touch). Then bake (plastic bag removed) it at a LOW temperature, 130 degrees or so, about the inside of a car with the windows up on a warm day - NOT HOT! To hot will turn it to Rawhide or babiche (sp) as the French Canadiens called it OR, even crack it.
http://www.knifenetwork.com/forum/showt ... hp?t=10270
The leather will soften with use again and if you are not an experiencened leather worker you risk making a suit of armor for your knife. Just Case the leather and letting it dry will work to some degree.
Water is both your friend and enemy - be careful.
EDIT: Get the leather too wet or leave it in the refrigerator to long you WILL GET MOLD. Then you will need to get a wood renwer containing Oxalic Acid to remove the mold - DON'T use bleach.
Wrap the knife in a single layer of plastic wrap and place it in the sheath before you start to dry. This way the sheath will shrink to fit the knife.
Good luck, it will either work or make a mess of your sheath depending on your skill with the leather.
Myself, I don't do it.