The story appears to be mostly true. It was told by the niece of one of the men who participated in the early experiments.
Note that these experiments were before there was even an experimental vaccine. They are still working on getting samples of the rabies virus. Any exposure to the rabies virus would have been a death sentence for the exposed person. It would have been a painful, protracted death, with a chance of infecting someone else along the way - friend, co-worker, or family.
The story is quoted in the Louis Pasteur biography by Patrice Debré.
From page 430, the author quoting from Dr. Roux's niece, Marie Cressac:

Dr. Roux's niece, Marie Cressac, has described the atmosphere of the
first experiments:
[Roux], Chamberland, and Thuillier bent down around a table. A large dog
was tied down on it, its muscles contracted and its fangs bared... If the
animal, despite all the precautions, had caused them to make a false move, if one of them had cut himself with his scalpel, and if a small piece of the rabid spinal cord had penetrated into the cut, there would have been weeks and weeks filled with the anguished question: will he or will he not come down with rabies?... At the beginning of each session, a loaded revolver was placed within their reach. If a terrible accident were to happen to one of them, the more courageous of the two others would put a bullet in his head.... They were no longer just "researchers" absorbed in the meticulous work of their laboratory; they were pioneers, adventurers of science.
There is a reference number (15) to a footnote that unfortunately isn't available in the online copy of the book. That footnote would presumably give a more accurate source for the quote.
Note: "Mostly true." The story as it is repeated around the internet mentions Pasteur himself keeping the gun at hand. The quote from those involved don't mention Pasteur actually having the gun in hand. It is all about his assistants who actually handle the infected animals.
I can't find anything that says that Pasteur himself really ever touched the weapon or was involved with it in any way.