Jewish Survey of Yesterday: The 1981 New York Jewish community survey

As I wrote about the Rochester study, my mind kept turning back to the 1981 study of the New York Jewish community. I reproduce my description of the study from my dissertation below.

Leo Tolstoy’s (2004 [1877]) opening to Anna Karenina, "All happy families are alike; each unhappy family is unhappy in its own way," applies well to survey research. There are a limited numbers of ways to get a survey right, but countless opportunities for serious error. In many ways, a post mortem of the abject failures among surveys is more instructive than recapitulating the successes. The 1981 New York survey is a study in contrasts. Where the Los Angeles, Denver, and Chicago studies either oversampled low density areas or maintained a constant probability of selection, the Federation of Jewish Philanthropies of New York (1984) continued its long tradition of abysmal methodology by limiting random digit dialed calls to the 40 telephone exchanges (i.e. XXX-NNN-XXXX) with the highest proportion of distinctive Jewish names. At most, these directories could yield 400,000 telephone numbers. In practice, the number of households contained therein was probably much lower, given that many telephone numbers go unused or are ineligible as they belong to businesses or government agencies. These calls were supplemented by a mail survey to a sample of households with distinctive Jewish names. It is unknown whether the Federation went to the trouble of removing duplicate cases. The study’s authors estimated that 31 percent of Jews had distinctive Jewish names. The study itself estimates the Jewish population at 1.67 million. Assuming, for the sake of argument, that this figure is correct, at most the study could have covered only 31 percent of New York Jews (c. 520,000) plus some fraction of less than 400,000 households. In other words, at least half of New York’s Jews--and probably many more--were systematically excluded from the sample. This would have been problematic enough if it were a random half being excluded, but clearly this was not the case. Those excluded lived in less Jewish neighborhoods and did not have identifiably Jewish surnames. It is hardly a leap of faith to assume that all estimates from the study were irreparably biased.

Because the study was not based on a universal sample, the Jewish population size could not be directly estimated by multiplying the proportion of Jewish households found by the survey by the known population of the study area. Instead, the population estimate is extrapolated from the numbers of households with distinctive Jewish names on cantilevers of guesswork and surmise. A contemporary study of Hartford Jewry, comparing estimates from a purely random digit dialed sample with the distinctive Jewish names approach, concluded that: "It seems clear that [distinctive Jewish name] estimates cannot be brought to a reasonable level (i.e. even to 'ballpark figure') by any kind of adjustments" (Abrahamson 1982).

The reason for this travesty appears to be the New York study’s vast sample size of 4,505 Jewish households, which it boasted was "the largest ever single study of a Jewish community outside the state of Israel" (Federation of Jewish Philanthropies of New York 1984:71). (Large as it was, the sample size was still well below the number of interviews conducted for the National Jewish Population Study of 1970-71.) Survey research can be seen as a balancing act between sample quality, sample size, and cost. The Federation of Jewish Philanthropies traded quality and/or cost in order to maximize sample size. The benefit of a large sample is that it decreases the size of the confidence intervals around an estimated value. This is, of course, pointless when the estimate will be biased as a result of the sampling scheme. The fact that the largest Jewish community in the world, with the densest Jewish population in the United States, and presumably the greatest financial resources could not mount a valid study at a time when the cost of survey research was probably at an all-time low beggars all description.

References

Abrahamson, Mark. 1982. "A Study of the Greater Hartford Jewish Population, 1982." Greater Hartford Jewish Federation, Hartford, CT.

Federation of Jewish Philanthropies of New York. 1984. "The Jewish Population of Greater New York: A Profile." Federation of Jewish Philanthropies of New York, New York, NY.

Tolstoy, Leo. 2004 [1877]. Anna Karenina. Translated by R. Pevear and L. Volokhonsky. New York: Penguin.