The Wikipedia article has a good introduction to this, the hygiene hypothesis:
Allergic diseases are caused by inappropriate immunological responses to harmless antigens driven by a TH2-mediated immune response. Many bacteria and viruses elicit a TH1-mediated immune response, which down-regulates TH2 responses. The first proposed mechanism of action of the hygiene hypothesis stated that insufficient stimulation of the TH1 arm of the immune system lead to an overactive TH2 arm, which in turn led to allergic disease. In other words, individuals living in too sterile an environment are not exposed to enough pathogens to keep the immune system busy. Since our bodies evolved to deal with a certain level of such pathogens, when it is not exposed to this level, the immune system will attack harmless antigens and thus normally benign microbial objects—like pollen—will trigger an immune response.
There are two things at work here: first, that the immune system cannot develop a sane reaction to those irritants with which it has little experience; and second, that the immune system scales its reactions according to the amount of irritants in the environment. If that amount decreases, it adjusts its expectations to assume that any irritants outside the normal, diminished range are much more severe than they, under ordinary circumstances, would be.
And shortly thereafter, relevant citations:
Studies have shown that various immunological and autoimmune diseases are much less common in the developing world than the industrialized world and that immigrants to the industrialized world from the developing world increasingly develop immunological disorders in relation to the length of time since arrival in the industrialized world. doi:10.1002/ppul.10323
Longitudinal studies in the third world demonstrate an increase in immunological disorders as a country grows more affluent and, presumably, cleaner. doi:10.1371/journal.pmed.0040070
The use of antibiotics in the first year of life has been linked to asthma and other allergic diseases. doi:10.1378/chest.129.3.610
Of course, while there is significant evidence in favour of this model as the cause of many forms of allergy, it does not account for all types: for instance, it is possible (and common) to develop allergies later in life despite regular exposure to the offending material.