No, the right half of the field of vision goes to the left brain, and vice versa. A quick search showed up a book by a Harvard professor with this quote:
A massive stroke in the left side of the brain leads to paralysis and lack of sensation in the right face, arm, and leg and to loss of speech. What is less commonly known is that such a stroke generally leads also to blindness in the right half of the visual world—the right visual field—
involving both eyes.
To test for such blindness, the neurologist has the patient stand in front of him, close one eye, and look at his (the neurologist's) nose with the other eye. He then explores the patient's visual fields by waving his hand or holding a Q-tip here and there and, in the case of a left-sided stroke, can show that the patient does not see anything to the right of where he is looking.
So the effect is quite easy to demonstrate scientifically (an awful lot of our knowledge of the brain comes, somewhat grimly, from studying what happens when you damage bits of it).
The reason they are reversed is actually one of optics, not neurology: light rays passing through a pinhole cross over, and project an image that is reversed both top-to-bottom and left-to-right. So it is actually the left half of each retina which transmits to the left half of the brain, and the nerve fibres which cross (at the "optic chiasm") are the ones closest to the nose.
