I couldn't find anything in the scientific literature, but then I thought I had found the answer.
I read the description of an experiment performed by the BBC show The Truth About Food. Not peer-reviewed scientific journal, but perhaps the best you can hope for on a question that is unlikely to get normal funding. It sounded promising.
Their video link is (currently) broken, but the segment is available on YouTube.
Unfortunately, it turned out to be a terribly low-quality experiment.
Three couples took part. (Flaw: sample too small to draw any strong conclusions.)
The men were given a diets free of garlic, asparagus and alcohol, but high in one of spices, fish or fruit. (Flaw: There was no control to see if the changes were non-existent.)
After three days, and kept in the dark about which diet was involved, the three women tasted the semen from a test-tube. (Flaw: There was no control; they relied on memory to compare; presumably with a faster and less formal presentation.)
They were given four choices (including one "distractor" answer) to select between. (Flaw: The hypothesis was that changing diet could affect the taste in some way. The actual test was that it would affect the taste in a manner that was previously associated with the food. It would fail to detect if fruit actually made the semen more spicy in flavour, for example.)
One of the three selected correctly - this is not significantly different to random guesses. (Do I need to do a detailed statistical analysis to prove this?)
The reporter's conclusion was entirely facile:
Evidence perhaps that food flavour molecules may pass into the seminal fluid.
I draw an entirely different set of conclusions:
The flavour of the semen is not strongly changed in a expected direction and in a way that shines through the unusual protocol (e.g. test tubes), to enable all three women to predict the diet correctly.
Even the BBC's Science and Nature unit has a pitiful understanding of experimental design.