Well, yes and no. The earliest biographies of St. Patrick do relate how he defeated druids and magicians, sometimes in supernatural warfare. The Vita tripartita Sancti Patricii has several stories like the following:
Enna saw the druids (magi) wishing to kill Patrick, and he said to his
son Conall, "Go and protect Patrick, that the magi may not kill him."
Patrick perceived them, and ethereal fire burned them, to the number
of nine.
But the much later legend about getting rid of "snakes" (which didn't exist in the first place) really is a story about a massive group of venomous creatures who were driven off a hill, as we learn in Jocelyn's 12th century Life of Patrick:
Even from the time of its original inhabitants, did Hibernia labor
under a threefold plague: a swarm of poisonous creatures, whereof the
number could not be counted; a great concourse of demons visibly
appearing; and a multitude of evil-doers and magicians. And these
venomous and monstrous creatures, rising out of the earth and out of
the sea, so prevailed over the whole island that they not only wounded
men and animals with their deadly sting, but slayed them with cruel
bitings, and not seldom rent and devoured their members. And the
demons, who by the power of idolatry dwelled in superstitious hearts,
showed themselves unto their worshippers in visible forms; often
likewise did they, as if they were offended, injure them with many
hurts; unto whom, being appeased with sacrifices, offerings, or evil
works, they seemed to extend the grace of health or of safety, while
they only ceased from doing harm. And after was beheld such a
multitude of these, flying in the air or walking on the earth, that
the island was deemed incapable of containing so many; and therefore
was it accounted the habitation of demons, and their peculiar
possession. Likewise the crowd of magicians, evil-doers, and
soothsayers had therein so greatly increased as the history of not any
other nation doth instance.
And the most holy Patrick applied all his diligence unto the
extirpation of this threefold plague; and at length by his salutary
doctrine and fervent prayer he relieved Hibernia of the increasing
mischief. Therefore he, the most excellent pastor, bore on his
shoulder the staff of Jesus, and aided of the angelic aid, he by its
comminatory elevation gathered together from all parts of the island
all the poisonous creatures into one place; then compelled he them all
unto a very high promontory, which then was called Cruachan-ailge, but
now Cruachan-Phadruig; and by the power of his word he drove the whole
pestilent swarm from the precipice of the mountain headlong into the
ocean.
It is interesting how these early hagiographies have been completely forgotten. Even Wikipedia repeats the confused muddle posed in this question about "snakes possibly meaning pagans". St. Patrick's tussles with druids are well-documented in sources dating to soon after his lifetime.
