A fee is only defensible if the client knew about it before they booked. Your terms are shown at booking and in the confirmation for exactly that reason. If the terms are vague, the argument on the day is unwinnable, no matter who is right.
When it happens, say it plainly and without apology: the terms, what is being charged, and what happens next. 'My policy is 24 hours, so the deposit stays on this one. I have got you in on Thursday at two — shall I put you down?' Naming the next appointment in the same message turns an ending into a rebooking more often than you would expect.
Do not argue the principle. If they dispute it, restate the terms once and stop. A client who wants to fight about a cancellation fee is not going to be persuaded by a third message, and the thread is your record either way.
Waive it deliberately, not weakly. A first offence from a good client, or a genuine emergency, is worth waiving — say that you are waiving it and why, so it reads as a decision rather than as a policy that does not really exist. What loses you the fee permanently is waiving it silently, because you have then taught everybody that it is negotiable.
Watch the pattern rather than the incident. One late cancellation is life; three from the same person is a client who should be on a deposit before their next booking.