Here's a link to the
Merck Vet Manual. The protocol is to put the suspected animal under observation for 10 days to see if it exhibits symptoms. If it does it is put down and a sample of gray matter is sent off for testing. Even if it has been vaccinated. If it's a stray then they jump right to testing.
That tells me ... it's not super critical to get a jump on this the same day. But definitely see a doctor as soon as possible.
I had a scare sorta like this as a kid, wrestled a mouse out of my cat's mouth and got properly bit by the little fellow. I'd just seen that Different Strokes episode with the rabid dog scare where they described the treatment for rabies and I was terrified. The doc told my folks over the phone not to sweat it and I'd be fine.
On the other hand, a cat rushing you unprovoked it aberrant behavior and a mouse biting the hand of its rescuer would be totally normal.