I've found a document with lots of references called, The Effects of Moderate Beer Consumption.
It contains this chart, that shows drinking 4 drinks a day for a man has the same relative risk of mortality as drinking none.
From Di Castelnuovo A, Costanzo S, Bagnardi V et al. (2006). “Alcohol dosing and total mortality in men and women: An updated meta-analysis of 34 prospective studies”. Archives of Internal Medicine, 166:2437-2445.
From a UKSheffield Addiction Research Group, on a UK Parliamentary website, about the evidence that the chart presents:
Studies of the relationship between alcohol and harm are frequently subjected to systematic reviews in order to identify those studies of the highest quality and to aggregate the effects into a more robust overall estimate of the relationship. Systematic reviews are carried out on a regular basis for individual harms and for all-cause mortality. This evidence is also compiled in the WHO's work on the burden of total disease which is due to alcohol. Therefore, the evidence on which guidelines are based can be considered as of the highest quality available and is reviewed and updated on a regular basis.
Many studies of the risk of alcohol consumption are based on survey data which ask respondents to report their consumption. Such reports are known to substantially underestimate the amount of alcohol believed to be consumed based on sales data, by between 40% and 60%. Although efforts have been made to explain and address this problem, many estimates of the risk from alcohol consumption may be biased upwards by under-reporting of heavy consumption.