What do we mean by weaker, or by stronger?
"Women really are stronger than men, according to study"
Is the title of a BBC article which says
A recent academic study has shown that under extreme conditions such as famines, epidemics and enslavement, women are able to survive for longer than men.
Across modern populations, women outlive men in almost all instances
The study says this
Women in almost all modern populations live longer than men. Research to date provides evidence for both biological and social factors influencing this gender gap. Conditions when both men and women experience extremely high levels of mortality risk are unexplored sources of information. We investigate the survival of both sexes in seven populations under extreme conditions from famines, epidemics, and slavery. Women survived better than men: In all populations, they had lower mortality across almost all ages, and, with the exception of one slave population, they lived longer on average than men. Gender differences in infant mortality contributed the most to the gender gap in life expectancy, indicating that newborn girls were able to survive extreme mortality hazards better than newborn boys. Our results confirm the ubiquity of a female survival advantage even when mortality is extraordinarily high. The hypothesis that the survival advantage of women has fundamental biological underpinnings is supported by the fact that under very harsh conditions females survive better than males even at infant ages when behavioral and social differences may be minimal or favor males. Our findings also indicate that the female advantage differs across environments and is modulated by social factors.
Now this type of strength might not be what the question refers to. Perhaps some people might prefer to label this attribute as (biological) toughness rather than strength?
At the very least, we should be aware that when one person says group A is stronger than group B or another person disputes that statement, we should consider carefully what type of strength is being referred to.
Types of Strength/Weakness
The question quotes a video as saying
The belief that women are somehow a naturally weaker gender ...
but it may be that the author of the video is, perhaps in part, perhaps mostly, referring to other kinds of historically perceived weakness: emotional and intellectual.
This answer doesn't address that but we probably shouldn't assume the author was only referring to muscle size and not to strengths like toughness, determination, perseverance and so on.
We can put the above concerns to one side and consider muscular strength.
Sexual dimorphism in primates
Age and gender comparisons of muscle strength in 654 women and men aged 20–93 yr in the Journal of Applied Physiology contains this graph
Regression analysis of age- and gender-related differences in concentric (Con; A) and eccentric (Ecc; B) peak torque of knee extensors at slow (0.52 rad/s) velocity. Both Con and Ecc peak torque declined significantly (all P < 0.001) for men (r 2 = 0.30 for Con and 0.19 for Ecc) and women (r 2 = 0.28 for Con and 0.11 for Ecc).
We can see that many women are stronger than many men (i.e. there is considerable overlap in the data) but that the average strength for women is lower.
Sexual dimorphism in Humans is relatively small according to anthropologist Clark Spencer Larsen of Ohio State University writing in PNAS
Humans today display relatively limited sexual dimorphism (≈15%), whereas
some of the other hominoids (gorillas and orangutans) are highly dimorphic
According to Kirchengast S.
Although sexual size dimorphism has a clear evolutionary basis and is caused by genetic and hormonal factors, socio-cultural factors such as gender role in society and gender typical workload influence the degree of sexual size dimorphism too.
So some of the differences we perceive may be the result of cultural forces of the sort which change over time and from place to place.
When is using averages (mean or median) useful and when inappropriate
The question asks about the strength of the average woman vs the average male but in most situations (e.g. interviewing job candidates) we are not interacting with the average person we are more likely interacting with a person whose physical characteristics may be anywhere on the spectrum above. If you select two people at random, one male, one female, there is a significant probability (less than .5 but far higher than 0) that the female is stronger than the male. Focussing on small differences in overall averages for a large population is inappropriate for most situations which involve only a few individuals.
It would be bizarre to say to a candidate, you are strong enough to do this job but you are ineligible because you are a member of a group whose average strength is lower than the average of some other group. However this is exactly the sort of discrimination that has occurred in the past and that is part of the context for the video which stimulated the question above.
We should note that the quotation in the question doesn't use the word "average".
It may be that the video referenced in the question is concerned whether this perception (true or not) contributes to an unreasonable bias against one part of our population.
The better question may be what do we do with this information. David Haye may be stronger than Stephen Hawking. Should we skew society to benefit people with large muscles at the expense of the others?