I’ve done fitness level swimming (quarter mile sets on a short course) while fasted and not had an issue, but like with any fasted exercise it’s going to depend on how hard you push yourself. I’m trying for endurance so i tend not to go at my max speed during the sets.Totally not grounded in reality but I always feel I can do tighter flipturns without food on my stomach.
I do keep a closer eye on signs of electrolytes imbalace though since dehydration while swimming is really sneaky.