A few days ago, a friend of mine posted a link to a column on Cracked.com regarding the misconceptions that people have about U.S. history. One of the issues discussed in the column was the tendency for most modern people to think of the pre-European contact native peoples of the Americas as both primitive and few in number. In the process of discussing the matter, though, the columnist routinely drew from population figures that seemed, at the very least, to be a bit inflated. This likely wasn't entirely the columnists fault - population estimates for prehistoric populations tend to vary widely depending on the methods (and sometimes motives) used by the person doing the estimates, and the author of the column (or, more likely, their source) appears to have used the highest population figures that they could find in order to bolster their arguments about the sophistication of Native American groups*.
So, I commented on the link, noting that the population sizes given seemed, to me, to be rather on the large size, and likely weren't very accurate. My friend responded by asking me to cite sources.
This is, of course, a perfectly reasonable response. Initially, my ego was a bit bruised (as I deal with analyzing these sorts of things for a living, I have grown accustomed to people just taking my word for things when, in fact, they should be asking more questions), but once I got over myself, I realized that this was a perfectly reasonable thing to ask.
The problem, however, is that there is no one good source to which I could direct my friend.
There are literally hundreds of published papers that provide prehistoric population estimates, usually providing detailed descriptions of the methodology used for making the estimates, and there are other sources that synthesize and summarize this information, though typically without providing a good description of the methods used to reach the various population estimates. But there are no good resources that summarize the difficulties of estimating populations, nor the generally unreliable nature of most estimates. This is something that is, quite simply, professional knowledge held by every archaeologist, and gleaned from spending several years in graduate school reading article after article, paper after paper, and book after book in which population estimates are made, and noting the various pros and cons of every method that you encounter.
In the case of population estimates, they are always proxy measures, as prehistoric peoples, by the very nature of being prehistoric (that is, not leaving behind written records) didn't leave us with a census. You can count burials in a cemetery, but this assumes that most of the people who died in an area were buried in its cemeteries (which is often not the case), that funerary customs and taphonomic conditions typically resulted in remains that will preserve archaeologically (often not the case), and that the burials provide a representative sample of the population (visit your local cemetery and note the volume of old vs. young people, as well as evidence of age associated with economic status, and you'll quickly see that this assumption usually doesn't hold). So, once you do your count, you have to make a number of assumptions in order to come up with a population estimate.
Similarly, you could identify an artifact or feature (say, for example, house remains) and estimate population based on that. The problem there, though, is that you have to make some assumptions regarding the number of people per artifact or feature, and these assumptions are often rather shaky. Even when they're well founded, they still require an assumption that the ration of people artifact/feature remained stable across space and/or time, which is often not true.
There are other methods, but what they all have in common is that we are trying to use some object or material that preserves archaeologically, and convert the number/volume into a population estimate. And to do this, we have to make a number of assumptions (the precise assumptions made vary depending on the specific proxy measure), and this is rather problematic. This is why archaeologists who are honest will both show their work and provide a range of possible population sizes (for example, John Johnson, in his doctoral dissertation examining prehistoric populations in the Santa Barbara Channel area, explained what his sources of data were, the shortcomings and benefits of each of those sources, and then provided a range of possible population sizes, rather than assert that a population of one particular size occupied the area - and then he followed that up by explaining that his estimates were the best that could be done with his data, but might well be wrong).
The problem, though, is that I have learned this by reading research papers, articles, and books, and from trying to apply some of these methods myself. It's not something for which there is a ready-made source to which I can point people. There was a decent article for the lay public published on Slate.com of all places that summarized some of these issues, but even it was rather simplistic in its descriptions, and exaggerated that guesswork side of things a bit.
This got me thinking about the sheer volume of other issues that my colleagues and I are aware of simply by virtue of our training or repeated professional exposure, but for which we can not point people towards one, or even a few, good sources to summarize the issues. These include issues ranging from the use of comparative ethnography in archaeological work; both the correct and invalid use of linguistic similarities in determining cultural affiliation; when it is valid to use old sources, and when it is a bad idea to do so; and when data patterns are due to human activity as opposed to natural movement and degradation of items over time within an archaeological site.
I don't know that there is a good way around this. One could certainly write a source and try to get it published (the publication part is a bit problematic for a variety of reasons that I am going to get into at the moment), but it would, by definition, be outdated almost as soon as it appeared, and it would only cover one specific subject, while there remain many others uncovered. On the other hand, I can routinely go into descriptions of these problems when I encounter them, but that may or may not satisfy individual people asking questions and I am only one person doing it.
So, it is something I will have to think about (and, if I am lucky, get some of my tenured academic colleagues to think about it - they get paid to write, whereas I do not). I am not convinced that it is unsolvable as a problem, but it is so far unsolved.
*The irony of this is that the population size estimates are not the most important elements to establishing cultural sophistication, but for some reason we tend to equate statements such as "settlement X had a population bigger than London's" with an assertion that the settlement was, therefore, in some (usually unclear) way more complex than London, when, in fact, all that has been claimed is that it was larger.