As pointed out in the comments, we have to distinguish two different concepts when talking about the relation between sex/gender and colors: the ability to assign different names to different shades (color naming) and the ability to perceptually distinguish shades with different physical attributes (color discrimination).
There are several studies that support the claim that females perform better than males with regard to color naming (this difference can probably best be described as the effect of a more refined technical vocabulary). However, the present question asks about color discrimination: Can women distinguish more colors than men?
It's a rather well-established fact that males have a higher risk of color deficiencies: About 8 percent of males and about 0.6 percent of females are affected by red-green color blindness. Most of these color deficiencies, in particular red-green blindness, are due to an inherited variations that are associated with the X chromosome. Thus, even though the phenomenon is more complicated than this simple account, at least the first part of what is stated in the video is true: information about light-sensitive visual pigments is stored in the X chromosome, and due to basic genetics, women are less likely to be affected by mutations of the pertaining sequences.
So, it's not wrong to claim that "[this] is why females can distinguish more colors and shades", but this claim is only true in a statistical sense – on an individual level, it doesn't follow that an average woman will be able to distinguish more colors than an average man if both are unaffected by any genetically-caused color blindness.
There seem to be only few studies that have investigated gender differences in color sensitivity with unimpaired subjects. One somewhat recent study reports some differences between males and females with regard to the perception of monochromatic light across the visual spectrum. Their first result concerns the translation of wavelengths to perceptual sensations:
[R]egardless of the particular hue, males required, on average, a wavelength 2.2 nm longer than the wavelength needed to elicit the same sensation from females.
Another result concerns the discriminatory ability of males and females, where the authors claim to find "systematic differences between the sexes":
While there are no statistically significant sex differences, the male and female curves are not identical. Applying Exploratory Data Analysis to these data: there appear to be systematic differences between the sexes. In the middle of the spectrum, males have a slightly broader range of relatively poor discrimination (540–560 nm for Newtonian-view; 530–570 nm for Maxwellian-view).
The study supports the notion that males and females do not perceive colors in exactly the same way, and that there may be a range in the visual spectrum where females have a better discriminatory ability than males. Yet, the authors acknowledge that both effects are relatively small ("There were relatively small but clear and significant, differences[…]"), and it's unclear how the findings, which were obtained explicitly in a highly experimental setting, translate to every-day color discrimination.
One every-day application of color discrimination that has been investigated in a small-scale study is the field of dentistry, where dentists routinely have to match the natural shade of their patients' teeth to that of dentures. As before, the participants in that study were restricted to those that were not affected by color blindness. The authors investigated the agreement between male and female participants when attempting to match the shade of teeth to shades on three standardized color schemes. If women are indeed "more capable than men regarding shade selection and color matching", i.e. if females outperform males in color discrimination, one may expected to find a higher agreement rate between the females than between the males.
However, this prediction was not borne out by the data:
Although it is still [in 1991] thought that "getting a second opinion on color interpretation from a female staff person is advisable since women perceive color more accurately than men in general," we found no evidence to support this statement. Men tended to agree with one another slightly more than women in shade selection.
A null effect of sex was reported in an early study using a standardized hue discrimination test: While color discrimination was found to deteriorate with increasing age, no significant difference between male and female participants was observed, with the exception of a single age range (15-24 years) where males reached a higher discrimination score than females (for what it's worth, the participants of the previous study on denture shades also fell within this age range).
I don't have access to the full text of another potentially relevant study that investigated color discrimination, color naming, and color preferences in a population of 80-year old males and females. The abstract mentions a gender difference for color naming, but not for color discrimination.
To summarize: Based on the sparse empirical record on the effect of gender on the ability to distinguish colors, there is weak evidence that males and females without color blindness perceive colors somewhat differently, and that there is a range in the visual spectrum where females show a slightly better discriminatory performance than males when using monochromatic light sources.
However, I couldn't find compelling evidence that this marginally significant performance difference results in a substantially better color discrimination by women as far as real-world objects (case study: shades of teeth) are concerned.
