This can be traced to a questionable 1922 anecdote.
Okay, I have an answer now.
The original French quotation, "le terrain est tout, le microbe n’est rien," was in vogue among doctors in the 19th century, as seen for example in a review of the 1885 book Leçons de clinique médicale by S. Jaccoud:
he becomes an eclectic sage, as we used to say, and insists perhaps too forcefully on the importance of the terrain: the terrain is everything, the microbe is nothing. ("il devient d'un sage éclectisme, comme on disait autrefois, et insiste avec trop de force peut être sur l'importance du terrain: le terrain est tout, le microbe n'est rien.")
Lyon médical, Volume 48, 1885
or in this 1902 German review:
Opposition to the doctrine that infection with the tubercle bacillus alone is sufficient to cause tuberculosis or even phthisis has visibly gained ground in recent years. Everyone, especially the practitioners, is working hard to prove that hereditary disposition is the actual beginning of the disease -- "que le terrain est tout, et que le microbe n'est rien". ("Die Opposition gegen die Lehre, dass die Infektion mit dem Tuberkelbacillus allein ausreiche, um Tuberkulose oder gar Phthise zu erzeugen, hat in den letzten Jahren sichtlich an Terrain gewonnen. Von allen Seiten, namentlich von den Praktikern, wird eifrig daran gearbeitet, den Nachweis zu führen, dass die hereditäre Disposition der eigentliche Beginn der Erkrankung ist, „que le terrain est tout, et que le microbe n'est rien".")
Aronsohn, Ed. "Beziehungen zwischen Tuberkulose und Krebs." DMW-Deutsche Medizinische Wochenschrift 28.47 (1902): 842–845.
By 1962 this phrase had become associated with the antivaxxer movement.
Vaccination is a method constantly denounced. It is, alongside [the loss of natural] bread, the other “major plot”... The general conception of health presented ... defends the idea of “natural immunization by purification of the internal environment: the microbe is nothing, the terrain is everything” (LVC, n° 172, March 1962). ("La vaccination est une méthode constamment dénoncée. C’est même, avec le pain, l’autre « complot majeur » ... La conception générale de la santé présentée ... défend l’idée « d’une immunisation naturelle par purification du milieu intérieur : le microbe n’est rien, le terrain est tout » (LVC, n° 172, mars 1962).")
César, Christine. « Les métamorphoses des idéologues de l'agriculture biologique. La voix de La Vie Claire (1946-1981) », Écologie & politique, vol. 27, no. 1, 2003, pp. 193-206.
So, who first attributed it to Pasteur? This information cannot be found on Google Books/Google Scholar but it is in BnF Gallica.
Another legend to be destroyed: Pasteur, affirm those ignorant of his work, saw diseases as the work of germs alone, and ignored the importance of the terrain that causes germs to develop. A famous novelist of the end of the 19th century, a great friend of Dieulafoy, even went so far as to claim that Pasteur, shortly before his death, confided to one of his close friends: “I was wrong, the microbe is nothing, the ground is everything”. Pure invention of the novelist! ("Une autre légende à détruire : Pasteur, affirment les ignorants de son œuvre, n'eut en vue dans les maladies que le germe infectieux, il ignora i’importance du terrain pour le développement du germe. Un romancier célèbre de la fin du xix^e siècle, grand ami de Dieulafoy, alla même jusqu'à prétendre que Pasteur, peu de temps avant sa mort, aurait confié à un de ses intimes : « le me suis trompé, le microbe n'est rien, le terrain est tout ». Pure invention de romancier!")
Bulletin de l'Académie nationale de médecine, 21 Nov 1946
Basically, this quotation says that the attribution to Pasteur arose at around the same time period that the phrase was in vogue. I think it is plausible that the original use of the phrase was a lazy shorthand describing French doctors who sought out what we might call preventative medicine, and was then exploited by a "novelist" falsely caricaturing Pasteur as a germ-crazy fool unaware of bodily conditions. Some decades later, this sort of caricature evolved into an entire pseudoscientific "terrain theory" although doctors never denied the importance of "terrain" in the first place.
The "novelist" in question is Paul Bourget. From Google Books, it looks like he initially published this anecdote in a 1922 issue of L'Illustration:
Professor Renon, who has just died prematurely, told me that, watching Pasteur during his last illness, the latter, whom he thought was asleep, had woken up from an indefinite reflection to tell him: "Renon, it is Bernard who was right. The germ is nothing. The terrain is everything." ("Le professeur Renon, qui vient de mourir prématurément, me racontait que, veillant Pasteur durant sa dernière maladie, celui-ci, qu'il croyait endormi, s'était réveillé d'une réflexion indéfinie pour lui dire : « Renon, c'est Bernard qui avait raison. Le germe n'est rien. Le terrain est tout. »")
This also appears word-for-word in his 1929 book Au service de l'ordre, on p.199. L. Renon was a real doctor at l'Hôpital Necker. However, according to an affidavit drawn up at the time of Pasteur's death, the attending doctors were named Emile Roux and Louis Vaillard.