Short answer: NO
Details:
There are a few problems with using a dictionary to estimate vocabulary size which Thorndike pointed out as early as 1924 in The Vocabularies of School Pupils. In short, using a dictionary will always result in an overestimation, and using bigger dictionaries will lead to even larger overestimations. This phenomenon is known as the big dictionary effect.
The testyourvocab site runs into related problems. Word selection is based on headwords, leading to the dictionary-based problems noted above. It also fails to use pseudowords to compensate for humans' uncanny ability to overestimate their own ability, a.k.a. illusory superiority. The inclusion of pseudowords on a checklist (yes/no) test is generally considered (e.g. Eyckmans (2004) Measuring receptive vocabulary size) to be the only way to make the test reasonably reliable because it ensures that a minimum level of word knowledge is enforced when claiming knowledge (i.e. one can really identify the form of the word).
There are other problems. Which level of word knowledge do you set as the cut off point for accepting that a word is part of someone's mental lexicon? Simply form identification (I've seen it before, but I have no idea what it means)? Full productive use (I can use this word appropriately in any context without hesitation)?
The project at vocabularysize.com is attempting to refine measurement techniques as well as establish norms. One unique aspect of this project is that they are trying to measure different aspects of word knowledge separately and then later combine them to come up with a more accurate measure of vocabulary size. For example, when testing the word nation, most people would assume that national, nationalistic, antirenationalisation, etc are also known because of normal morphological processes involved in creating the related words. However, morphological awareness grows just as vocabulary size does.
Here's an unlikely example to illustrate: if two people know the same 5,000 base words, the one with the ability to recognise the most common 100 affixes in English is going to have a smaller vocabulary size than the one who knows the most common 200 affixes in English. This is an oversimplified example because it implies that measuring word types (unique forms) is more valid than word families (base word + inflections + derivations), which is not a particularly useful way of measuring, but you get the idea. One's mental lexicon can't be measured just in terms of 'how many words'. It must be measured by looking at a variety of aspects dealing with knowledge of form, meaning, collocation, register, syntactic roles & constraints, etc.
So, to answer your question directly, no. There are too many problems defining concepts such as word and know to confidently state something like "There are x number of words in your vocabulary". What is more likely is to rate someone's vocabulary abilities, in terms of recognising various aspects of word knowledge or using those aspects productively, against age-based norms. There's also interesting work in psycholinguistics with reaction times in lexical decision tasks which hint at the interconnectiveness of one's mental lexicon which could also be used to rank subjects. Both approaches, though, would be more akin to standardised test scores which serve to rank rather than measure absolutely.
how was such a number arrived at?
There are a variety of methodologies, most centred on sampling from dictionaries which is unreliable. Despite early knowledge of this problem (Thorndike 1924), vocabulary size tests which have sprung up over the last 100 years have often tended to use this methodology. This is one reason why there are so many estimates which vary so widely.
A good overview of the issues involved in accurately measuring vocabulary size can be found in Nation, I.S.P. (1993) Using dictionaries to estimate vocabulary size: essential, but rarely followed, procedures. Language Testing, 10(1), 27-40. The most important points are that great pains must be taken to avoid oversampling from higher frequency words which tends to happen when using dictionaries as a source.
Is the "20,000 words" claim (or whatever it happens to be) just a guesstimate or are these numbers arrived at in a more mechanical and, ideally, reproducible procedure?
Some tests are more rigorous and valid than others. Measuring the vocabulary size of infants and very small children, for example, has seen very good gains in precision and accuracy, mostly because a fairly complete list of all words they could possibly know can be constructed. This becomes impossible as one learns more words. Most claims of adult vocabulary size are based on reproducible procedures, but the variance in estimates can usually be traced back to differences in defining the concept of word and the level of word knowledge tested. Remember, early IQ tests were basically vocabulary size tests, so there are many attempts out there which have norms, validation studies, etc. Many of them are quite useful as a ranking or discriminating measure, but are less useful to answer the more academic question of absolute vocabulary size.
Have then been any formal studies in this vein?
Yes, lots. Some of the first were:
- Holden, E. S. (1875). On the number of words used in speaking and writing. Bulletin of the Philosophical Society of Washington, 2, 16-21, 28.
- Kirkpatrick, E. A. (1891). Number of Words in an Ordinary Vocabulary. Science, 18(446), 107–108.
Other important studies include:
- D’Anna, C. A., Zechmeister, E. B., & Hall, J. W. (1991). Toward a meaningful definition of vocabulary size. Journal of Reading Behavior: A Journal of Literacy, 23(1), 109-122.
- Dupuy, H. J. (1974). The rationale, development and standardization of a basic word vocabulary test. US Govt Printing Office, Washington DC.
- Schonell, F. J., Meddleton, I. G., & Shaw, B. A. (1956). A study of the oral vocabulary of adults. Brisbane: University of Queensland Press.
- Seashore, R. H. (1933). The measurement and analysis of extent of vocabulary. Psychological Bulletin, 30, 709-710.
- Seashore, R. H., & D., E. L. (1940). The measurement of individual differences in general English vocabularies. Journal of Educational Psychology, 31, 14-38.
I've got a list of 30-40 studies, so I won't bore you with the rest.
In closing, perhaps the reason why measuring vocabulary size and debates about it have persisted for so long is that early IQ tests were essentially vocabulary size tests and people still tend to equate knowledge of arcane or obscure words as a sign of intelligence or other socially desirable traits. Although there is some correlation between vocabulary size and other traits, the strength of that correlation is often not as strong as people would assume, and it also doesn't imply any sort of causality.
Full disclosure: I'm part of the vocabularysize.com project
