According to this Health Library article:
A few studies have shown a decrease in
sperm concentration and motility when
subjects wore tight-fitting underwear
compared to loose-fitting underwear
(such as boxer shorts). It is worth
noting that most of these studies were
conducted on very small numbers of
subjects. One study published in the
Lancet included nine subjects; another
published in Reproductive Toxicology
studied only two subjects.
However small, both of those studies
concluded that wearing tight-fitting
underwear does increase scrotal
temperature enough to significantly
reduce sperm count and motility.
Neither of these studies, though,
suggested that the deleterious effect
on sperm was long-lasting.
One of the largest studies supporting
the claim was conducted on 50 men at
Leibig University in Germany. Its
results indicated that high scrotal
temperatures (defined as near body
core temperature) negatively affected
spermatogenesis (the development of
mature sperm cells). The German study,
published in Human Reproduction,
measured scrotal temperatures on men
wearing cotton jockey briefs, cotton
boxer shorts, or no underwear, and
compared the temperatures after the
men completed physical activity
(walking for 45 minutes on a
treadmill), and after they sat in a
fixed position for 45 minutes. The
study found significantly higher
scrotal temperatures while the
subjects wore jockey briefs versus
boxers, regardless of activity level,
and the lowest scrotal temperatures
when the subjects wore no
undergarments.
It lists references at the bottom of the article.
So it looks like there are studies that support the claim which answers your question. However, the numbers are significantly low to make a solid claim.