You did miss that Harry was used to dodging frying pans, which means it happened with some frequency. Obviously, Dumbledore would have had no clue, as Miss Figg lived blocks away. Still it did happen as evidence by canon. BTW I would note it is Physical Abuse even is the swing misses.