Several studies or press articles suggest that keeping a cell phone in a male's trousers pocket (and cell phone usage in general) may harm that men's fertility (increase sperm issues):

I think most of these sources are pretty old, and kind of vague about the exact protocol (i.e. was the phone in a call or waiting for one?).

Some other sources suggest that more studies are needed or that the risk is not obvious:

Is there a real risk?

Side question: I thought that a non moving cell phone waiting for a call was almost passive, that is, not emmiting continuously but mostly "listening the signal". Is a phone still harmful in this context?