Take the 2-minute tour ×
Skeptics Stack Exchange is a question and answer site for scientific skepticism. It's 100% free, no registration required.

So I watched the following 1 minute long Youtube clip from CSI New York. In the clip, using what seems to be the recording from a standard bank camera, they zoom in at least 100, and see the image of the culprit in the reflection of the eye of the girl.

Now, I thought this was completely ridiculous, so much so that I thought it was actually really funny.

However, my friend argued that there are very good tricks for image enhancement, such as "super resolution" a procedure where multiple frames of a video to produce a much higher single resolution image. He did think the show bends the truth quite a bit, but how much?

Honestly, I don't actually know anything about these things, so my question is:

How good is modern image resolution enhancement? Also, how far off are the CSI television programs?

Thank you,

share|improve this question
4  
the part where they got the reflection of the basketball from is less than a square millimeter the eye is not that good of a reflector regardless of how good your camera is, the angle of the closeup eye is straight on instead of a bit top down. check tvtropes on how ridiculous it can get in media (they even reference the scene you linked) –  ratchet freak Nov 9 '11 at 21:51
1  
on a side note the wiki for super resolution has some examples but those use a dozen frames for double the resolution –  ratchet freak Nov 9 '11 at 23:03
1  
I always like the example from Blade Runner, where they not only zoom in but track around. Not just enhancing, but creating :-) –  Rory Alsop Nov 10 '11 at 9:24
2  
It is totally real. Check out this video for another example: youtube.com/watch?v=gF_qQYrCcns ;-) –  ESultanik Nov 10 '11 at 13:00
1  
Maybe more appropriate for photography.se –  Ebenezer Sklivvze Nov 10 '11 at 13:16

3 Answers 3

up vote 21 down vote accepted

Short answer: you can obtain some very good results, but only under certain conditions and absolutely not even close to what is shown in the linked video clip.

My company, Amped Software, develops image and video processing software for forensic and intelligence applications, so basically we are the real world counterpart of the CSI software.

With reference to the general problem of quality enhancement, I can tell you that for our market it is a huge problem to live up to the expectations created by TV series and Hollywood movies. You can see on our samples page that sometimes the results we are able to get are really amazing, but it is important to understand that we can obtain them only under some conditions: if there is information that is covered by disturbs, but it is there, we are able to recover it. If there is no information, we can't and we must not recreate it. In this particular application is essential not only getting the results from a visual point of view, but also following a scientific workflow that must be accepted by the court.

Last year I presented a research describing issues and results on almost 200 cases I've worked on and the final result was the following:

  • in more than 50% of the cases there is nothing to do (for example recovering a license plate that is 5x2 pixels is completely impossible with any software on the world);
  • in about 30% of the cases we can get some little result (for example restoring some letter of a license plate or improve the overall appearance of a face);
  • in 10% of the cases you get good results (you get most of the license plate, for example).

Please note that all these cases had severe quality issues. If their quality was good, we weren't asked to work on them.

For what regards specifically resolution enhancement:

  • when you zoom on an image you are interpolating missing pixels: from a single image you can improve visually the appearance of the image but you will not add any real detail;
  • super resolution techniques may yield good results under certain conditions: you should have enough frames, shifted by a non integer amount of pixels and preferably with few compression artifacts. In the best case you can expect good results within 2x and 3x zoom.

What is shown in the video clip can be possible only if the original video has been shoot at several megapixels and then you will have the resolution to zoom very close (more or less like you do on Google Maps). Of course, at that point there still would be other problems, like the right focus, low light condition, the fact that the perspective of the eye is different from that of the whole subject in the video, just to mention a few.

share|improve this answer

There are multiple algorithms to improve image quality. Of course none of them are as spectacular, as in CSI and movies.

As already mentioned, for video multi-frame super-resolution algorithms might be applied. But for generic case it provides very limited improvement. Basically you might expect 2x improvement, but not 100x zoom as in CSI.

However, you might achieve spectacular results if some assumptions about the image being enhanced are made. For example if you know that an image fragment is a face, then specialized algorithm takes in account typical shape of human face. If you assume it's printed text, other specialized algorithm matches to particular alphabet. These algorithms use stochastic methods. Thus basically it's no longer 100% certain information, it's a guess with certain degree of probability.

Also, with these specialized algorithms most typically you don't actually get spectacularly enhanced image, but rather match against database of known faces (for facial recognition) or recognized text (for text/license plates).

TL;DR: given very low resolution image fragment it might be possible to tell if it's the suspect's face. But there is absolutely no way to do it in cool visual way they show it on CSI.

share|improve this answer
1  
Just as a side note: resolution enhancement is better when we're talking about microscopic scale. Super-resolution microscopy allows one to increase resolution over the diffraction limit (that is, the physical resolution limit due to the optics of a microscope). That, of course is something that requires specifically designed machines and a special way of capturing the image, you can't apply those tecniques to a random low-res image. –  nico Nov 10 '11 at 11:52

This is actually related to another field of my expertise, astrophotography.

There are certainly limitations in what can be digitally enhanced from blurry photographs. It's not uncommon for amateur astronomers to "over-enhance" an image they've taken such that the "sharpen" tool in their image processing software creates details that were not in the original. For example, you can sharpen an image of Saturn enough to make it appear as if you've observed the Encke gap with equipment less than the 24" Ritchie-Cretien-on-top-of-a-mountain-in-perfect-conditions that would be required to actually record the image.

There is software (Registax) that can create a clearer image by selecting the clearest frames of a movie captured by a video camera and adding them together. But the fact of the matter is that the best professional telescopes in the world were not able to get the kind of detail that the tiny telescopes of the Voyager probes got. Which is why the enormous amount of money was spent on such endeavours.

If photographs could be enhanced to anywhere near the detail that TV crime dramas suggest (and produce evidence that could be used in court), then two things would be true: blurry security camera footage would never be shown on TV to catch criminals, and the billions-of-dollars Hubble Space Telescope (and by extension, the much larger James Webb telescope in the works) would be completely unnecessary.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.