You could be over the amount of calories your body needs to lose weight. If your goal is to stay as you are (in shape as mentioned) not gaining sounds pretty good?
You could increase your fasted time, 16:8, 18:6, 20:4, and/or decrease the amount of full fat milk you drink. Even a (slow, if needed) switch to lower fat milks would help decrease the overall calories.
If you drink full fat milk during your fasting period, that breaks your fast.
You still need to count calories when doing IF to lose weight.If you cant exersize then you might need to eat a bit less than someone who does.If its not too personal what medical condition do you have because theres usually some way to work around them.Also it dosent sound like you need to lose that much weight id be careful to approach this in a healthy manner. I was severely anorexic as a teenager because I was a little overweight and ended up becoming obsessed with losing as much weight as possible.