Don't soak it in oil. That will further deteriorate the leather.
Cut the leather spacer to fit over the tang and make it as close to the outline of the handle as you can. Make only one slit in the leather.
Epoxy it in place, and once it has cured, grind/file/sand the washer down-sanding/filing, etc in one direction only until it is flush.
Color-match with leather dye if necessary.
Bill DeShivs, Master Cutler
Factory authorized repair for:
Hubertus, Grafrath, Ritter
Schrade Cut. Co., Geo Schrade, Pressbutton, Flylock