You should really be getting back whatever quantity of oil you put in. It seems as a starting point that you are using to much Methanol (i really hope you aren't using Menthol ). 20% by volume of oil is all you need. So 8ltrs for 40ltrs oil.
Make sure you are allowing the reacted fuel enough time to settle out completely after reaction, as you may be taking good fuel with the Glyc when you drain it if you drain too early. Overnight is generally sufficient.
Thirdly, are you titrating? If you guess and add too much Caustic or Potassium you will get more Glyc which is a waste if it was uncalled for.