Survival of old atDNA segments

[Frederator]

Recombination is an extremely variable process, but we do have reliable information about the statistical averages and standard deviations to make some kind of educated guess about how much of a given ancestor's atDNA should remain intact among their descendants worldwide at a given number of generations. The calculation is highly dependent on the assumed rates of natural increase, so although this will probably never have a clinical application, it is an interesting thought experiment from which users can develop an expectations about how many matches, relatively speaking, they should have from a specific ancestor. If nothing else, the experience of observing the different results obtained by using different assumptions is a gateway into applied history and demography.

I'm new at this myself, so it would be interesting to see whether any other users with formal training in these fields have much to add. But I start setting my baseline expectation around Malthus' assumed maximum rate of natural increase (i.e., births minus deaths, excluding in and out migration), which allows a population to double in the span of about 1 generation. I think this is based on a very specific set of population ratios re: gender and age, but allowing for practical purposes no significant barriers to family formation, so while this rate has been approximated several times in world history, I don't think it is possible, even in theory, to maintain it over a prolonged period of time. For calculation purposes, I use a factor of about 2.52 every 30 years.

I'm still learning about the conventions surrounding the determination of underlying assumptions, so I expect my results to be very crude, although hopefully broadly correct, in a comparative sense, with regard to specific historical populations. At this point, I'm not doing much more than comparing elapsed time to the changes in the estimated total population by various authorities, and try to reduce my assumptions about migration to the barest, most simple possible. Possibly I could improve my methodology without undue difficulty by looking at high-level changes in life expectancy, but to date I haven't tried this.

One phenom I am particularly interested in using the resulting figures to examine is the observed tendency of atDNA donors from certain ethnic or socially self-defined populations to experience a high volume of very strong matches attributable to very remote relationships, at least compared to the FTDNA customer base, which seems to be primarily the USA, maybe people with significant colonial-era descent. There are a number of such populations of interest defined to date, but I think the one that I've seen most often discussed in these forums are Ashkenazi Jews, so that's where I began my review.

Very detailed, very reliable information about the population size and composition is simply not existent for most of the critical period, so I have necessarily had to make a best guess from a range of disputed estimates. Maybe people with more information on the history of this population or demographic methods in general can offer suggestions or alternatives, but I think what I've done is not bad for a first attempt.

Using the figures implied by the current version of the following Wikipedia article, I've used worldwide population totals of 1 million, 6.2 million and 17 million for the years 1750, 1900 and 1939, respectively.

https://en.wikipedia.org/wiki/Historical_Jewish_population_comparisons

This results in estimated factors for natural increase of about 1.44 and 1.57 through 1900 and 1939, respectively. Not strictly comparable, given the complexities of migration, but maybe still worth reviewing, is my calculated average natural increase factor of about 1.10 for the population within the boundaries of modern Germany between 1600 and 1830. No doubt that is a vast understatement of the rate of increase for the descendants of German-speaking people worldwide, as the rates of natural increase for the United States, even those states like Pennsylvania, where German immigration was especially heavy, approached the Malthusian limit, I'm guessing a factor of around 2.33 during the late colonial period. Germany seemed as good a starting point as any, since I believe that self-reported census figures show this to be the single largest descent group in FTDNA's primarily US customer base.

http://www.tacitus.nu/historical-atlas/population/germany.htm

https://en.wikipedia.org/wiki/Demographic_history_of_the_United_States#Immigration

https://books.google.com/books?id=aTYrDwAAQBAJ&pg=PA663&lpg=PA663&dq=%22natural+rate+of+increase%22+%22colonial+america%22&source=bl&ots=pJu3tq181o&sig=ACfU3U2K26Rt2cSbPFMIGS-mmlCJaPOg3g&hl=en&sa=X&ved=2ahUKEwjRltLf6u7hAhUDlawKHc0OCYYQ6AEwBHoECAkQAQ#v=onepage&q=%22natural%20rate%20of%20increase%22%20%22colonial%20america%22&f=false

So all, in all, considering the relatively small proportion of the German population who could financially afford to emigrate, and what I think were the atypical and exceptionally low barriers to family formation temporarily available in colonial America, I'm going to guess that the world Jewish population had experienced rates of natural increase that were significantly higher than the typical European population, and for a much longer period of time. For the purposes of this pilot study, let's say an average factor of about 1.5 per generation since 1600, as compared to maybe 1.2 for a typical European population.

Okay, now I launch into the application for atDNA analysis.

Using these figures, and assuming a 50% loss of intact ancestral contribution per generation through recombination, I would expect, on average, that the most remote ancestor to contribute an intact contiguous 10cM segment to be born about 15 generations ago for a person of primarily Ashkenazic descent, vs. about 12 generations ago for a person of primarily other European descent. In other words, I would guess that a person of primarily Ashkenazic descent would, on average, have about 8 times as many atDNA matches as a person of primarily European descent; that is, 15-12=3; 2^3=8.

That is maybe a minimum differential, since I guess in the case of endogamous populations, there is the heightened possibility of a "rejoining" of ancestral segments to resemble an ancestral segment inherited fully intact. Methodologically, I suppose the next step in refining this estimate would be to calculate estimates of typical consanguinity over time, but I'm not sure it'd be worth the effort, at least at this early stage, given what seems to be the high odds against such a perfect fit given the sheer size of the genome. This effect, while real, is probably not statiscially significant.

And, of course, not all European populations are equally represented in the FTDNA customer base, with particularly low figures, I think from Italy, Greece, and points south and east. So to the extent my analysis has any validity at all, it's probably to people of northwestern European descent.

Just first thoughts. Anyone have their own?

[Frederator]

. . . . In other words, I would guess that a person of primarily Ashkenazic descent would, on average, have about 8 times as many atDNA matches as a person of primarily European descent; that is, 15-12=3; 2^3=8. . . .

Well, a chain is only as strong as its weakest link. Although I'm pretty happy (so far) with the rest of the reasoning and implementation of my calculation, I have to admit that this last bit, simply taking 2 to the power of the # of generations between some arbitrarily determined cM threshold, is not acceptable. I was hoping some reader would suggest a better, more methodologically sound option that was still simple enough to reduce the number of subjective input figures to an acceptable minimum. I didn't want to make this unnecessarily complicated. But it is a complicated topic, so maybe that was a pipe dream.

Anyway, I've given it another try, structuring it as an iterative process of arriving separately at an index of the typical number of matches for a person of primarily NW European descent, and then the same for a person primarily of Ashkenazic descent, and finally comparing the results to one another. Again the required inputs include an assumed rate of average natural increase specific to the donor's ancestral background, but this time I've also required an estimate of the average consanguinity among the donor's ancestors, considering the importance of this factor to endogamous populations like the Ashkenazim.

I haven't been able to identify any benchmark data about consanguinity for either population, although it seems like somebody probably has been gathering this data, given that Gedmatch.com has an entire utility devoted to this very topic. Right or wrong, for working purposes I've gone with estimates of TMRCA between Ashkenazi couples at maybe 6 generations (i.e., roughly 1/5 chance that any randomly selected couple will register as a match to one another) vs 10 generations for NW Europeans (i.e., 1/1,282 chance of a match). Why did I choose these? No idea. Pure uninformed subjectivity. I just had to start somewhere in order to work through the mechanics of the calculation. I would really welcome anyone else's educated guesses.

My first pass through returned a ridiculous ratio of nearly 26 Ashkenazi matches for every 1 NW European match, using the respective estimates of natural increase of 1.5 and 1.2, as per my last post. Correct me if I'm wrong, but I regarded that as ridiculously high. Is it because I got the rate of increase wrong or because I got the consanguinity estimates wrong? Who knows? My gut told me that I assigned too high a rate of natural increase to the Ashkenazi population, so I pared it down a bit, to 1.4. That returned a ratio of about 9 Ashkenazi matches for every 1 NW European match. Is that still to high? Maybe some reader will have an informed opinion, but at least it sounded closer to the ballpark.

This newer iterative calculation model also returned some interesting estimates of the typical ages of matches for each population. As you would expect, not only was the total volume of expected matches much higher for Ashkenazi people higher, but the age of matches also skewed much older. That's what the attached charts depict. The curve for Ashkenazi matches is much wider, with a far less pronounced peak, which I think makes sense given the anecdotes I often hear from Ashkenazi people of extreme difficulty in judging the likely degree of genealogical relationship simply by segment size or shared volume of atDNA.

I'm told that a lot of atDNA testing services have a so-called 'Ashkenazi filter' available to help such customers better research matches, and no doubt these require an estimation process analogous to the one I have just described. There must be people out there with an educated perspective on the problems I've discussed here, even if issues of proprietary information prevent them from sharing much of it.

[Frederator]

Sorry. Here are those charts. I got the display parameters wrong on one of them, and the edit function wouldn't display the new copies correctly.





Estimated NW European vs. Ashkenazi match distribution (by # generations removed).pngRatio - Est. % of matches by # generations removed, Ashkenazi to NW Europe.png

[Frederator]

I'm sure this is a controversial estimate, and at this point I don't really have enough understanding of the methodology to have much of an opinion on its trustworthiness, but at least this study offers data that I can use to improve my model's structure.

https://www.livescience.com/47755-european-jews-are-30th-cousins.html

Earlier I had developed a very basic model to determine the probability that a segment was 'legitimate', given the reported # of shared SNPs, and I can leverage this now to estimate the level of genetic diversity in the Ashkenazi population. At least within the range of SNPs tested by FTDNA.

That model assumed a perfectly even distribution of base values at each location, which would imply no assumed pedigree collapse. So by comparing the # of ancestors implied in a 31 generation timespan without pedigree collapse to the 350 ancestor founding population estimated by the article, and assuming 700 valid SNP matches define the theoretically valid segment, I come up with an estimated effective population genetic diversity of about 92% out of 100%.

I'll have to think long and carefully about the implications for matching, but it suggests that I'm going to have trouble reconciling the # of convergent matches with my assumed rate of natural increase. In other words, I think this is really going to mess with my calculation of probability to match.

[Frederator]

. . . . My first pass through returned a ridiculous ratio of nearly 26 Ashkenazi matches for every 1 NW European match, using the respective estimates of natural increase of 1.5 and 1.2, as per my last post. Correct me if I'm wrong, but I regarded that as ridiculously high. Is it because I got the rate of increase wrong or because I got the consanguinity estimates wrong? Who knows? My gut told me that I assigned too high a rate of natural increase to the Ashkenazi population, so I pared it down a bit, to 1.4. That returned a ratio of about 9 Ashkenazi matches for every 1 NW European match. Is that still to high? Maybe some reader will have an informed opinion, but at least it sounded closer to the ballpark. . . .

This discussion is a little misleading and I'm not sure it has any practical value. Now that I review the substance of my calculation, what I'm really saying is that these ratios apply to the entire current population, including people who have not even performed atDNA testing. There is probably no practical way to test how representative any given database is of the world total via some sort of empirical observation. Which is too bad. I really wanted some kind of check figure.

[khazaria]

Problem! There were many more than 350 genetic ancestors of the modern Ashkenazic population, it's just that those extra ones, who joined after the initial founders, were not ethnic Ashkenazim but rather Sephardic Jews, Sicilian Jews, Italki Jews, Mizrahi Jews, Poles, and Spaniards - and these extra ones are only ancestors of subsets of Ashkenazim rather than the whole population.

[Frederator]

Thanks for the tip.

This summary is very brief, and therefore necessarily ambiguous on some important points. But I think it strongly implies that this effective 350 person founder population was ethnically mixed in the way you are cautioning me.


"Further, that group of Jews who experienced this "bottleneck" was of approximately evenly mixed Middle Eastern and European descent."

https://www.livescience.com/47755-european-jews-are-30th-cousins.html

Do you have much insight into their methodology? Like, maybe did they do a survey of the genome of people in our contemporary population and compare the diversity observed to this basic theoretical model assuming no pedigree collapse and work backwards from there to infer the effective (surely NOT literal) population size? Or maybe to a broader sample of people from a variety of ethnic backgrounds?

I would like to use these insights to apply the resulting model to a variety of other ethnic groups, without the need for costly/time consuming genetic surveys. Like maybe using historical demographic data. My basic model suggests that, at zero population growth and no assumed pedigree collapse, ancestral contribution is expected to completely disintegrate within 15 generations. Maybe I need to apply a buffer for statistical deviation, but my first thought is that a useful estimate for any model of consanguinity within any ethnic group could be reliably performed using demographic estimates extending back ~30 generations, which, maybe not coincidentally, is how far this study reached back.

It would be neat if they performed both analysis to validate both methods.

[Frederator]

. . . . Do you have much insight into their methodology? Like, maybe did they do a survey of the genome of people in our contemporary population and compare the diversity observed to this basic theoretical model assuming no pedigree collapse and work backwards from there to infer the effective (surely NOT literal) population size? . . .

I'm sure that literal population size was much larger, as I believe you imply. But since I'm only interested in this demography in relation to the atDNA matches registered among living donors, I'm pretty sure the effective population is the statistic I want to focus on.

So now that I think about it, some sort of survey of the diversity among living donors for genetic diversity must be the primary methodology the study employed. But it also implies that any attempt to infer diversity for other populations on the basis of demographic history may be a lot more complicated than I would like.

[khazaria]

That study relates to the common pool of Jews in medieval Germany/Austria/Bohemia who initially formed the Ashkenazic culture. Yes, they were already mixed by that time (Middle Eastern + Southern European + also minor North African elements), about 30 generations ago.

The problem is that study ("Sequencing an Ashkenazi reference panel supports population-targeted personal genomics and illuminates Jewish and European origins"), unlike the later study "The time and place of European admixture in Ashkenazi Jewish history" by some of those same scientists, implies that those are the only people whom modern Ashkenazim descend from when they use phrases like "The Ashkenazi Jewish (AJ) population is a genetic isolate" and "the narrow AJ bottleneck, of just a few hundred individuals".

My autosomal DNA research during 2014-2019 has shown the widespread contribution of Sephardic Jews (but often different Sephardic ancestors for different Ashkenazim) and the less widespread genetic contributions of other types of non-Ashkenazic people, indeed within the past 15 generations.

[Frederator]

I think we're coming at this from two entirely different perspectives. As one should expect with a topic as complex as this.

I'm a generalist, not particularly interested in defining every narrow subset of this population. I'm only interested in the broadest, most commonly accepted, least-restrictive definition of Ashkenazic population in order to model the top-, population-level typical effects on atDNA matching. I naturally expect that more-restrictive definitions or focus on specific subsets of this population would return different results.

Unless you're saying that the term 'Ashkenazic' is meaningless, or that the specific sample this study examined is not representative of a reasonably broad interpretation of the term 'Ashkenazic', I honestly don't have a problem with this study.

I see that you have a great deal of specialist knowledge, and maybe are more interested in constructing a comprehensive anthropological history of this population than I am. You may raise the legitimate point that, even if the sampling was conducted in a way appropriate to a very general scope, there will be significant variation among individuals or specific subsets within that population, but I am interested only in a broad, population average. I understand there will be variation, but I am more interested in the most typical values.

[Frederator]

That study relates to the common pool of Jews in medieval Germany/Austria/Bohemia who initially formed the Ashkenazic culture. Yes, they were already mixed by that time (Middle Eastern + Southern European + also minor North African elements), about 30 generations ago.

The problem is that study ("Sequencing an Ashkenazi reference panel supports population-targeted personal genomics and illuminates Jewish and European origins"), unlike the later study "The time and place of European admixture in Ashkenazi Jewish history" by some of those same scientists, implies that those are the only people whom modern Ashkenazim descend from when they use phrases like "The Ashkenazi Jewish (AJ) population is a genetic isolate" and "the narrow AJ bottleneck, of just a few hundred individuals".

My autosomal DNA research during 2014-2019 has shown the widespread contribution of Sephardic Jews (but often different Sephardic ancestors for different Ashkenazim) and the less widespread genetic contributions of other types of non-Ashkenazic people, indeed within the past 15 generations.

Okay, I think I see your point now, but I'm not sure. Are you saying that I am applying my model incorrectly by moving from an a-historical effective genetic population figures to a very historical, census-derived figure to determine the likely # of individuals within a modern atDNA donor's pool of potential IBD maches? Because that observation is extremely relevant to my goal.

I don't think, however, that it will have a major impact on my end resulting estimates, though, just due to the fact that recombination washes people out of the pool pretty quickly, and I think I have some historical census data on population and age at first marriage that can allow me to perform some reasonableness checks on my results. Even at the implied level of reduced genetic diversity in the Ashkenazi population (writ large), I should be able to compare the results from my model for about 97% of the relevant range.

Here's what I mean: I have a most recent estimate of the worldwide Ashkenazi population of about 11.2 million in 2013. The study I linked to implies an average generation span of about 25 years. Let me know if you have a different opinion, but I've seen what are purported to be census figures from Poland and Russia, where the majority of Ashkenazi lived between the 18th and 20th centuries, showing average age at first marriage gradually increasing from the late teens to the mid 20's, making a 25 year average reasonable. Backing, then from 11.2 million over 10 generations, and using the implied average rate of natural increase of about 1.39742 from the study, my model estimates an Ashkenazi population of about 394,000 in 1763.

Against this, the data allegedly from a 1772 census of Poland showed about 309,000 Jewish inhabitants, from which one writer suggests implies a maximum Ashkenazi population of around 400,000.

https://en.wikipedia.org/wiki/Histor...on_comparisons

If you can accept the accuracy of this 400,000 estimate, and the reasonableness of my model's calculation of about 394,000 for roughly the same time period, I think it's a wash. The study we're talking about suggests an average wash-out period (i.e., average # of generations it takes for a given ancestor's contribution to disintegrate through recombination) of about 14 generations, and that about 97% of the atDNA matches such a person could get would relate to the most recent 10 generations, or about the period covered by our little census experiment above. The difference between the 350 person effective genetic founding population discussed in this study and the much larger actual historical founding population I think you imply could be due to a whole host of reasons like bottle necks, changes in the rate of natural increase, etc. that relate to the 22 generations outside the range of reliable census data. Probably just an application of the Pareto principle, and mostly not relevant for my specific purposes.

I hope this is the right track, or if not that you can clarify. I think I got a little confused by your references to the much larger diversity of the historical founding population as compared to the effective founding population inferred from genetics by the study. I'm really focused on the opposite end of the historical timeline.

[Frederator]

I'm not sure I fully grasped the importance of some of Khazaria's observations until just now. I think that the points he's raised will require me to significantly alter my model, but maybe not in the way I first suspected. I got caught up in the discussion of the ethnically mixed origins of this population, which I think is mostly irrelevant to my purposes. However, the distinction between an implied effective genetic founding population size and the actual, historic founding population size is extremely relevant to my purposes, though not in a way that renders the study useless.

In order for my model to make reasonable estimates of the expected quantity of atDNA matches for modern Ashkenazi people, I need to have reliable historic figures for population during say the last 15 or 20 generations, and the typical level of genetic diversity within the population. The summary of this study definitely does NOT provide the historic population figures, but does give me the information I need to calculate genetic diversity.

In my last post I basically argued that my then-current model was defensible because my population figures checked for the period relevant to my purposes. I now think that was incorrect, due to a fundamental misunderstanding of proper demographic methodology. The length of a generation shouldn't be defined as the span between the age at first marriage from one generation to the next, but rather the span between the midpoint of the reproductive life of one generation to the next. The generation length of 25 years implied by the study, I think, is wrong for my purposes, unless someone can provide a compelling reason for deviating from a 30 year convention.

I came to this conclusion after pondering the significance of the so-called Malthusian upper limit to the rate of natural increase I noted at the start of this thread. I wanted to figure out exactly what the rate of natural increase implied about typical age and gender structure and family size of a society at a given point.

. . . . I'm new at this myself, so it would be interesting to see whether any other users with formal training in these fields have much to add. But I start setting my baseline expectation around Malthus' assumed maximum rate of natural increase (i.e., births minus deaths, excluding in and out migration), which allows a population to double in the span of about 1 generation. I think this is based on a very specific set of population ratios re: gender and age, but allowing for practical purposes no significant barriers to family formation, so while this rate has been approximated several times in world history, I don't think it is possible, even in theory, to maintain it over a prolonged period of time. For calculation purposes, I use a factor of about 2.52 every 30 years. . . .

I found that by multiplying the implied annual rate of increase by 400, and subtracting from 30, I could usually arrive at a pretty close approximation of the sex averaged age at first marriage. And by subtracting from 40 and dividing by 2, I could usually arrive at a decent estimate of average completed family size. This seems to be due to the limits imposed by the biological process itself, requiring 2 partners and a fallow period after childbirth, and a reproductive life that spans from about age 20 to age 40--midpoint at age 30. Naturally societies with younger marriages tend to have more children and grow faster.

I say "usually" because the rate of natural increase is really the net difference between births and deaths, and absent some catastrophic event like a war or a prolonged negative economic shift, the number of deaths is typically trivial in comparison to the number of births. At least since the Middle Ages. Although mostly irrelevant for my purposes, gender skewing in certain atypical social structures could theoretically be significant for specific ethnic or national groups.

Anyhow, what this implies is that my population estimates for some recent generations is probably much too large. Possibly it may not effect relative comparisons of estimated typical atDNA match quantities between ethnic groups, but I tend to doubt it. I'm pretty sure the reason my backwards-looking estimates are too high is because my model neglected significant mortality events (e.g., the Holocaust).

I think this puts more emphasis on the importance of selected check figures, like the 1772 Polish Commonweath census. I would like to see more documentation on this, and support for why some authors seem to feel that this particular subset formed a particularly large subset of Ashkenazim.

[Frederator]

Ugh. This is going to be painful. I had assumed that such an intensively studied population as Ashkenazi Jews would have developed some clear consensus about recent demographic history at this point, but this seems to be far from the case. That 400,000 estimate for the Ashkenazi population in 1772 seems far too small given this excellently reasoned discussion of their pre-20th century distribution and the practical factors affecting population growth.

https://www.academia.edu/33380156/The_origin_of_East_European_Ashkenazim_via_a_southern_route

At least this entry from the Jewish Encyclopedia gives me a large variety of data to work with. I think I'll probably start from the 1900-01 and 1897 censuses and extrapolate backwards from there given some more reasonable estimate of population growth from the previous article.

http://www.jewishencyclopedia.com/articles/13992-statistics

These articles also raise the question of how methodologically appropriate it might be to consider central and western european Ashkenazim in the same study with eastern Ashkenazim. But given the ratios of population size, maybe the effect will turn out to be minimal.

[Frederator]

Well, I've kicked the tires, and I think I'm well on my way to straightening out the demographic information.

The key seems to be threading the path between what I think are some fairly reliable data points:

-The Jewish Encylcopedia's census figures for 1897-1901
-Two reasonably consistent estimates of the population of European Jews around 1700
-Jits van Straten's observations about the shared demograhic constrains with the non-Jewish population

I think the figures returned by my model now mostly make sense. They're broadly consistent with demographic principles, anyhow, though I expect they may not meet approval of that school that insists on really dramatic rates of natural increase. They suggest, on average, an age at first marriage of about 27 over the relevant period and a completed household of about 6 children, reaching a maximum of 8 only in the 19th century. The only way to square this with all the anecdotes I heard of exceptionally young marriages would be if there were a surprisingly high rate of ordinary mortality in the Ashkenazic population as compared to the national average, and I'm not sure how I would research that.

I should probably revisit my model's analysis of the non-Jewish population and whether I really understand the implications of the Columbia University for consanguinity. But at the moment, my figures suggest that an Ashkenazic atDNA donor today should expect roughly twice as many matches as a donor of other, primarily NW European descent. Does this reflect actual experience? I'm not sure I have a super-firm expectation, since I'm not sure about how representative the database composition is.

There are still some other kinks I have to work out. There is an 11% discrepancy between my model and the census returns for 1897-1901, but I think that may just be a trivial timing difference with no significant impact on my net results. I guess we'll see.

[khazaria]

Frederator, it seems like you're doing good work with the demographic research. We also have to remember that Ashkenazim were early and enthusiastic adopters of DNA testing in part because Family Tree DNA's founders are Ashkenazim and because I started recommending this company within webpages of interest to the Ashkenazic community since 2000 and advertising them as an affiliate of theirs since 2002. The many millions of Americans of Greek and Italian descent weren't as big into this during most of this timespan, by comparison. So, we have to remember that part of the reason there are so many Ashkenazim in the Family Finder, Y-DNA, and mtDNA databases is they (we) got a head start and it kept accelerating. It's not only because of endogamy. On top of that, Ashkenazim get even more matches because of the huge participation of Americans of Latin American descent who have partial Sephardic ancestry, which again is partly due to things Bennett and I have been doing to discuss and promote DNA testing among them.