Can the human eye distinguish frame rates above 60 Hz?

Question

I know that frame-rates above 60fps all look the same to the human eye. Is that true? Why? If so, why do graphics cards boast anything higher than that?

Answer 1

Eyes? No. Humans? Yes.

You'd be hard pressed to get 60fps out of the human eye.

In laboratory conditions, it takes around 150 ms for neurons in the visual system to begin to recognize and categorize a newly appearing visual input.

However, this little factoid is not the frame rate specification for human vision.

If real-world perception were to follow this same pattern, then for a considerable time after each saccade we would still be perceiving the old retinal input, rather than the information currently on the retina. In fact, we should have to wait around 150 ms to ‘see’ what is in front of our eyes after each saccade, by which time the oculomotor system has already begun to choose the next saccadic target.

That would suck.

Fortunately, the human eye is more than a camera* with fat pipe connection to the brain.

While holding a pen, for example, the sensory input is limited to the receptors of a few fingers, leaving the majority of the surface of the pen outside of our direct sensory range. Nonetheless, we perceive a complete object, not a pen with holes where our fingers do not touch. Similarly, our visual system actively perceives the world by pointing the fovea, the area of the retina where resolution is best, towards a single part of the scene at a time.

Human vision does not have properties like frame rate, latency, resolution, et al.

Visual constancy can also be viewed as a temporal phenomenon: objects appear to be continuously present over time. Yet the duration of external events are typically longer than that of a single sensory ‘sample’ such as a fixation. Although movements of the eyes, head and body disrupt our steady access to these objects and events, the stream of consciousness continues smoothly across these sensory disruptions. This is an amazing feat, given that each saccadic eye movement creates a temporal disruption in the flow of information from the retina to higher perceptual areas. The motor smear on the retina during the saccade is suppressed, making us largely unaware of the retinal stimulation during this time period. In addition, each saccade requires the visual system to ‘re-perceive’ the information from a new fixation.

Time is relative...

...perceived time seems to shift forward, towards the beginning of the new fixation, essentially compressing the time immediately before and during the saccadic eye movement. One possible interpretation is that space and time are inextricably linked in the brain, with the pattern of strange perceptual effects reported for stimuli flashed around the time of saccades reflecting a spatio-temporal transformation between fixations.

---

The Bottom Line...

Human vision is not bound by frame rate.

---

Source of all quotes: Visual stability, References

More...
Looking ahead: The perceived direction of gaze shifts before the eyes move
How Human Vision Perceives Rapid Changes
From eye movements to actions: how batsmen hit the ball
Vision and the representation of the surroundings in spatial memory

*Actually comparing the human eye to a camera is like comparing a thermonuclear weapon to a pen-knife.

Answer 2

Yes, the eye can distinguish frame rates above 60 Hz. So can the brain. We are just not normally aware of it.

Conscious perception of flicker is measured in laboratories using the critical flicker frequency (CFF) threshold, which is the lowest frequency of flickering light (Hz) that produces the appearance of steady light in an individual. It's a probabilistic statistic that is estimated by testing an individual -- it depends on the individual and the testing methods. And the same person will have a different CFF depending on factors including fatigue.

Here is a pretty typical CFF plot (from Hartmann, E., B. Lachenmayr, and H. Brettel. "The peripheral critical flicker frequency." Vision research 19.9 (1979): 1019-1023.) showing how one person's CFF (the y-axis) varies at different visual eccentricities (the x-axis, greater eccentricities meaning looking at it less and less directly) and lighting levels (the different points and lines.)

As you can see, this person is around 60 Hz CFF in bright illumination when the flicker is around 15 visual degrees away from the center of their vision. This suggests that many people should be able to see the flicker of CRT monitors with 60 Hz refresh rates. They can. (As others have noted here, this is not possible with LCD monitors because LCD monitors do not flicker.)

Why isn't CFF even higher at these peak conditions? Good question. The bottleneck in consciously seeing flicker is not necessarily the human eye or even the thalamus or the cortex. The eye can transmit flicker well above 60 Hz to the thalamus and cortex. Lots of neurons can fire well above 100 Hz. And we can even measure neural response to high frequency flicker. Here are a few papers doing this:

Herrmann, Christoph S. "Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena." Experimental brain research 137.3 (2001): 346-353.:

It is also known that neurons in visual cortex respond to flickering stimuli at the frequency of the flickering light. .... We performed an experiment, where ten human subjects were presented flickering light at frequencies from 1 to 100 Hz in 1-Hz steps. The event-related potentials exhibited steady-state oscillations at all frequencies up to at least 90 Hz.

Williams, Patrick E., et al. "Entrainment to video displays in primary visual cortex of macaque and humans." The Journal of neuroscience 24.38 (2004): 8278-8288:

Macaque monkeys: Of 92 cells in the primary visual cortex exposed to a 100 Hz refresh, 21 (23%) significantly phase locked to high-contrast stimuli.

Humans: Responses measured using scalp EEG were seen at 72 Hz in some, but not all, observers.

Krolak‐Salmon, Pierre, et al. "Human lateral geniculate nucleus and visual cortex respond to screen flicker." Annals of neurology 53.1 (2003): 73-80.:

The first electrophysiological study of the human lateral geniculate nucleus (LGN), optic radiation, striate, and extrastriate visual areas is presented in the context of presurgical evaluation of three epileptic patients (Patients 1, 2, and 3). ... A periodic sinusoidal activity was recorded in Patient 1 at contacts lying in the LGN and optic radiation vicinity (G2 and G3) in the checkerboard reversal paradigm... Its frequency was 70Hz, identical to videodisplay refresh rate... In the same checkerboard paradigm, a similar periodic sinusoidal activity was recorded in Patients 2 and 3 at occipital electrode contacts (O1 to O6) exploring the calcarine area. It was not present in the other explored structures, especially not in the fusiform gyrus (contacts F1 to F6 in Patients 1, 2, and 3

So if the information is in the brain, why can't we perceive it? We don't really know. The simplest theory might be that these signals are just too weak. But it seems unlikely that we could record something with electrodes on the scalp that isn't strong in the brain.

It is also worth noting that our ability to perceive flicker is a side effect of our ability to perceive motion. Most motion perception takes place in situations where we have other information about the moving objects. Flicker perception as measured with CFF threshold or as noticed with monitor refresh rates is a strange edge case. It likely made little evolutionary sense to optimize this ability when we could already, for example, see most fast moving natural objects well enough to catch them. So you might invert the argument and ask why should we humans have bothered to see fast flicker?

Perhaps the most interesting possibility is that this may be a limitation of consciousness itself. An initial intuition is that conscious perception will likely be slower than the low-level processes it relies on. Because different types of perceptual processes have different computational demands, they take different durations to compute. Thus, combining perceptual processing in consciousness may be slowed by rate limiting steps. Alex Holcombe wrote a nice paper a few years ago summarizing the literature on temporal limits on visual perception of different kinds of stimuli. Moreover, Rufin VanRullen and Christof Koch have argued that conscious perception occurs in discrete batches. They don't think it is perfectly regular, but rather quasi-periodic and determined by the task at hand. Still, they are essentially arguing that consciousness itself has a refresh rate.

Answer 3

The Wikipedia article on Frame Rate suggests, IMO, that 60 Hz is not really the far end of the spectrum.

Major institutions such as Snell have demonstrated 720p72 pictures as a result of earlier analogue experiments, where 768 line television at 75 FPS looked subjectively better than 1150 line 50 FPS progressive pictures with higher shutter speeds available (and a corresponding lower data rate).

However the link to the cited article is broken.

Also, from same article, and more relevant:

Higher frame rates, including 300 FPS, have been tested by BBC Research over concerns with sports and other broadcasts where fast motion with large HD displays could have a disorientating effect on viewers. Whitepaper link

A higher FPS will also yield a natural motion blur, something that is usually computationally expensive to render, even at a lower FPS.

Answer 4

The human eye is not able to distinguish between 60 Hz and 100 Hz when only watching (even less could suffice provided the image is prepared correctly, as evidenced by movies projected at 24 Hz in the theather), but there is a distinction when interacting - you can notice the response is faster even beyond 60 Hz. This is augmented by the fact response time (latency between the input and the displayed image) is often several frames because of pipelined nature of the rendering technology, therefore the latency of 60 Hz game is typically over 60 ms, as evidenced by Gamasutra Measuring Responsiveness in Video Games.

Answer 5

Critical flicker frequency (the rate at which you can no longer distinguish a flickering light from a constant one) isn't everything. Research during World War II and afterwards using a tachistoscope demonstrated that people can identify airplanes or make correct shoot/don't shoot decisions based on images seen for as little as 1/100 of a second.

Answer 6

There are noticeable quality improvements when going above 60Hz.

Many televisions support 120Hz or higher conversion of a 60Hz input signal to make motion appear smoother: https://en.wikipedia.org/wiki/Motion_interpolation

One of the founders of the company behind the Oculus Rift claims that there are noticeable improvements when going above 60Hz for VR applications: http://www.pcgamer.com/oculus-founder-palmer-luckey-thinks-30-frames-per-second-is-a-failure/

Ultra High Definition video (8k) is specified to use 120Hz for improved motion quality: https://en.wikipedia.org/wiki/Ultra-high-definition_television

How much difference a higher frame rate makes appears to depend on the display and source material.

Answer 7

I once pressed a gamer about why he tried to boost his framerates into hundreds (severely sacrificing picture quality, of course). In the end it turned out it was because some games couldn't properly handle extreme framerates and exploits started appearing (such as enemies not hearing your footsteps and the like).

Of course, this does not apply to all games or all people, but it is one reason for trying to squeeze as much as possible out of your VGA.

---

To make this less anecdotal I did some Googling. Here's an analysis of Quake 3's 125fps jumping bug. Apparently similar bugs are carried over to other games which use an engine derived from Quake's. Here's another analysis both for Quake3 and Call of Duty, which focuses on jumping and running sounds. The magical 333fps count stands out.

While googling I came upon many forum threads, and I gather from those that official tournaments have FPS limits to prevent players from using these exploits. However many threads mentioned that "everybody is doing this" which suggests that for simple game servers this is a widespread practice. This is quite understandable, because if one person does this, then others get an unfair disadvantage.

Answer 8

If a viewer's eye is stationary with respect to an image-display apparatus, the viewer's ability to distinguish a particular frame rates from an infinite frame rate will fall off sharply as rates increase beyond roughly 12Hz (the rate at which many animated cartoons are drawn). If, however, the viewer's eye is moving with respect to the apparatus or the image shown thereby, such motion will cause each part of the image shown by the apparatus to be focused on different parts of the retina at different times. Even if any particular part of the retina would be unable to detect that the image focused thereon changes dozens or hundreds of times per second, the moving eye as a whole may achieve a much greater apparent temporal resolution.

As a simple example, if one's eyes track the motion of an object seen through a dark picket fence, different parts of the object will be visible at different times, but persistence of vision will result in all the pieces into a combining to form a consistent image. If instead one were to use a stationary 60fps video camera to record the object through the fence, the camera wouldn't be able to capture nearly as much information, and a person watching the video would have no way to see any details of the object which the camera never captured.

Further, as a more general principle, discrete spacial or temporal sampling is prone to introduce artifacts which may end up being visible no matter the sample rate. Suppose uses a 60Hz camera to shoot a video of an object which is flickering 29 times per second. There is no way for a 60Hz video to show 29 uniform-looking flashes per second. If one ups the video rate to 120Hz, that would make a 29Hz object look better, but a 59Hz object would then cause trouble. If one ups the video rate to 1200Hz, the 59Hz object would look fine, but a 599Hz object would cause trouble. If one were to record at 1200Hz but then add a 1/60 second blur to everything, one could pretty well ensure that any frequency of flashing light would either be slow enough to appear as a uniform flash or fast enough to appear as a uniform solid illumination, but one would need to record the video at a rate which is significantly more than twice as fast as the highest frequency of interest.