The question needs to be answered on a case-by-case basis. Certainly, not all GMO food is toxic, and the process of GMO does not inherently lead to toxicity. On the other hand, there is no guarantee that novel gene combinations can never lead to increased toxicity for individual organisms.
Below I have collected some sources, either neutral ones, or focussing on the risks. I did not include information from sources with a commercial interest in GMO, such as Monsanto, as I am of the opinion that information from corporate sources deserves to be distrusted.
A recent review study
According to Domingo et. al. (2011), who did an overview of studies into this question, the number of references concerning human and animal toxicological/health risks studies on GM foods/plants was very limited.
Domingo, J. L. & Giné Bordonaba, J. Environ. Int. 37, 734–742 (2011). Weblink.
The article is behind a paywall, but the abstract summarises:
An equilibrium in the number research groups suggesting, on the basis of their studies, that a number of varieties of GM products (mainly maize and soybeans) are as safe and nutritious as the respective conventional non-GM plant, and those raising still serious concerns, was currently observed. Nevertheless, it should be noted that most of these studies have been conducted by biotechnology companies responsible of commercializing these GM plants
This review was from 2011.
United Nations Environmental Programme assessment
The International Assessment of Agricultural Knowledge, Science and Technology for Development, an international effort co-sponsored by FAO, GEF, UNDP, UNEP, UNESCO, the World Bank and WHO, issued a lengthy report assessing what the title says. This includes (but is not limited to) issues related to GMO.
Several reports, as well as summaries, are linked from the United Stations Environmental Programme IAASTD page. Specifically on the safe to consume for humans issue, they write on page 200 in the global report (emphasis theirs):
The safety of GMO foods and feed is controversial due to
limited available data, particularly for long-term nutritional consumption and chronic exposure.
Food safety is a major issue in the GMO debate. Potential
concerns include alteration in nutritional quality of foods,
toxicity, antibiotic resistance, and allergenicity from consuming GM foods. The concepts and techniques used for
evaluating food and feed safety have been outlined (WHO,
2005b), but the approval process of GM crops is considered
inadequate (Spök et al., 2004). Under current practice, data
are provided by the companies owning the genetic materials, making independent verification difficult or impossible.
Recently, the data for regulatory approval of a new Bt-maize
variety (Mon863) was challenged. Significant effects have
been found on a number of measured parameters and a call
has been made for more research to establish their safety
(Seralini et al., 2007). For example, the systemic broad spectrum herbicide glyphosate is increasingly used on herbicide
resistant soybean, resulting in the presence of measurable
concentrations of residues and metabolites of glyphosate
in soybean products (Arregui et al., 2004). In 1996, EPA
reestablished pesticide thresholds for glyphosate in various
soybean products setting standards for the presence of such
residues in herbicide resistant crop plants (EPA, 1996ab).
However, no data on long-term consumption of low doses
of glyphosate metabolites have been collected.
For the sake of completion, they define GMO (Genetically Modified Organism) as An organism in which the genetic material has been altered anthropogenically by means of gene or cell technologies..
Union of Concerned Scientists point of view
The Union of Concerned Scientists is a an advocacy organisation, and not necessarily objective. However, I do think their assessments are evidence-based, so I think their point of view is worth quoting:
So far, scientists know of no inherent, generic harms associated with GE organisms. For example, it is not true that all GE foods are toxic or that all engineered organisms are likely to proliferate if released into the environment.
But specific enginereed organisms may have specific harmful effects by virtue of the novel gene combinations they possess. This means that the risks of genetically engineered organisms may vary widely, and therefore must be assessed on a case-by-case basis.
Specifically about toxins, they write:
Production of New Toxins
Many organisms have the ability to produce toxic substances, which help to defend them from predators. Some plants contain inactive genetic pathways leading to toxic substances, and new genetic material introduced through GE could reactivate these pathways or otherwise increase the plant's production of toxic substances. This might happen, for instance, if on/off signals associated with an introduced gene are located on the genome in places where they could turn on the previously inactive genes for producing the toxins.
Other studies
More recently, Séralini et. al (2012) conclude that a particular kind of genetically modified food is harmful to rats, but only on timescales similar to the rats lifetime, which would suggest that effects in humans may only become visible on a timescale of decades. The work by Séralini is highly controversial. He has been accused of fraud and his articles have been described as debunked; whether this is true or a sign of the debate itself being toxic, I don't know. Wikipedia has a whole article Séralini affair, which links further to himself and his articles. They're too controversial to base any conclusions upon, really.
An advocacy group called Earth Open Source has published a report GMO Myths and Truths, where GMO is understood as genetically engineered crops. This document specifically collects material critical of GMO, and is therefore not objective by itself, but it does build upon peer-reviewed publications and the authors have PhDs in relevant fields (molecular genetics and biochemistry). They list a number of studies linking how GMO's can be toxic, similarly to the note by the Union of Concerned Scientists.
Concluding note
"Safe to consume" or "harmful to humans" can relate to direct or indirect effects. Not all GMO food is automatically unsafe, but there exist secondary effects. For example, some GMO adoption leads to increased pesticide use, and there exist ecological risks which can indirectly affect human health. For further reading, these issues are addressed by the Union of Concerned Scientists, the UNEP, and the Earth Open Source report.
The scientific debate continues.