Iffy Match Lists: Why does GD/TiP fail to account for incompatible Haplogroups?

[Rheiner]

My kit has almost 1800 matches at the 12 marker level.

But many -- if not most -- are not true matches at all. And this seems to be a problem built into the FTDNA algorithm for matches.

I hand-sorted and checked each of about 300 BigY/TerminalSNP "matches": appx. 80% are not even in my U152 haplogroup; they are L21, DF27 -- even U106. As I understand haplogroups, there is appx. a 0% chance that any of these "matches" are viable.

Still, I just got an email from FTDA about a 12-step match: "After you log into your kit, the first step is check if any of your Y-DNA matches share your surname. If they do, examine their family tree or email them to look for common ancestors."

With the algorithm listing so many false matches, this seems like a fool's errand: It's pointless to email folks in different haplogroups. Isn't it?

What's more, at the 67 level, my kit matches four BigY individuals -- but only one is downstream of U152! (Others are L21) TiP reports for all of these questionable matches seem wildly misleading.

Would love for someone to explain why STR match lists can't factor in major upstream haplogroups. As it stands, the algorithm is a colossal time-waster, and a recipe for burnout.

I know that haplogroups are always in flux to a degree, as SNPs are sorted and placed. It's a dynamic science -- but the match lists aren't using all the available data, are they?

Thanks for considering.

(BTW: I have ordered my BigY upgrade. But lots of folks can't spend that much, and rely on the match lists. For them, the match-list seems shoddy and wasteful.)

You can check my work here.

[John McCoy]

It's not that an algorithm can't be devised to take into account all available information. Rather, the fact is that the algorithm just doesn't do that. I think most of us gave up on looking at Y STR matches below the Y-67 level a long time ago.

Some men happen to land in a group where the very close matches all have the same surname. Those are very lucky men! Their branch of the haplotree happened to be well established before permanent surnames became fashionable, and nearly everybody descended from that branch ended up with the same surname. For the rest of us, however, Y DNA has been far less informative in a genealogical sense, apart from helping us rule out ancestry from other families who share our surname but who are not on the same haplotree branch. And the matches below the Y 67 level, and even those at Y 67 that are not extremely close, have not helped us even do that much.

[Rheiner]

Thanks for that, John.

The word "match" is misleading in this context, considering the instructions given customers from FTDNA.

Perhaps they will work on a better algorithm -- or at least better instructions. I'll add that there's no reason to have confidence in TiP calculations. (False probabilities yield false hypotheses.)

My genealogical interest is 9+ generations ago, so I'm casting a broad net.

[DWFlineage]

Despite being an adoptee, I have been able to confirm my paternal & maternal line, and I am one of those "lucky men" mentioned by John, so I have close 67 & 111 marker matches. My 109/111 match is my great nephew, my birth father was his paternal great-grandfather, and we only mismatch by one value at dys390 & dys710, both fast moving markers. The tip indicator estimates 66% probability for a common ancestor in 0-4 generations, so I do question why the algorithm does not take into account that these two strs are fast moving?

[Rheiner]

Thanks for that. What's chiefly misleading about the TiP are its claims of "precision."

But your story is outstanding.

[DWFlineage]

Thanks for that. What's chiefly misleading about the TiP are its claims of "precision."

But your story is outstanding.

Thanks, amazing stroke of good luck that my great nephew decided to test at 23andMe where I also tested, and he is still my closest match there, 778cMs on 23 segments, so he agreed to test at FTDNA as well and he matches me the same here, as well as the ydna 109/111. My birth father has passed, and I am his youngest child. I will not get into specifics regarding who my birth mother was, but I have confirmed who she was via genetic scientists analyzing my autosomal dna, and my dna matches. I started this quest in 2009, so I only confirmed both my birth parents within the past year or so.

[Jim Barrett]

The word "match" is misleading in this context, considering the instructions given customers from FTDNA.

If your Y-DNA12 STR's have the same values then they match. Nothing misleading about that.

[Rheiner]

We differ then.

If you had bothered to click on the link above, and examine the spreadsheet of 12 STR matches, you would see something that might surprise you. If 80% of "matches" are in different haplogroups, it's not a match list, it's a fantasy list -- because if haplogroups don't match, it's misleading to instruct customers to pursue genealogical leads wholesale based on that list of "matches."

It's irresponsible. Should be corrected.

[DWFlineage]

We differ then.

If you had bothered to click on the link above, and examine the spreadsheet of 12 STR matches, you would see something that might surprise you. If 80% of "matches" are in different haplogroups, it's not a match list, it's a fantasy list -- because if haplogroups don't match, it's misleading to instruct customers to pursue genealogical leads wholesale based on that list of "matches."

It's irresponsible. Should be corrected.

I have many 12 marker matches, some with my surname most not. FTDNA staff has always explained to me that you need at least 67 or 111 marker results to get closer related men. I disagree with you that it is misleading, the 12 marker matches are matches to a common ancestor that could be recent if still a match at 67 or 111 marker level or could be 2000 yrs ago, but still a match.

[Rheiner]

No. These are very old subclades. If a "match" means a relation 4000 years ago, it's meaningless as genealogy. FTDNA should write instructions that reflect that. Check the data at the link above, please?

[bartarl260]

No. These are very old subclades. If a "match" means a relation 4000 years ago, it's meaningless as genealogy. FTDNA should write instructions that reflect that. Check the data at the link above, please?

Oh, you mean something like this chart? (Probability of relatedness on a Y12 STR match)
https://www.familytreedna.com/learn/dna-basics/ydna/

Probability that your common ancestor lived no longer than this number of generations ago.

Genetic Distance	50%	90%	95%
0	7	23	29
1	17	39	47

And you know what, I'll fully agree that those odds are more than a bit optimistic based on SNP testing outcomes. Even going with a 40 year generation(which is possible for men), 23 generations "only" takes you back 920 years. STR Convergence is a pain that way when you match with people on Y-12 where SNP testing indicates it's actually more than five thousand years ago. I have some GD0 matches like that [email protected]

But seriously, you're complaining about a test methodology where they give you a 1 in 10 chance of not being related in the past 23 generations, and how you're not going to be able to find a paper trail for that? Most people are tickled pink if they can reliably get back 7 generations on most family lines.

Or put it a little more on the nose. FTDNA's numbers, as quoted above, give a 5% chance of a 12 marker match of being more distantly related than 29 Generations ago.
You say you have over 1,800 matches. 95% works out to about 1 in 20. one-twentieth of 1,800 is 90. So by FTDNA's own numbers, you should have 90 "blatantly wrong" mismatches showing up on your match list when only using the 12 marker STR matching criteria. Of those 1800, an additional 90 of them are likely to falling in the 23 to 29 generation window, good luck on those paper trails.

Y25 has no statistics, Y37 is still on shaky ground at this point:

Genetic Distance	50%	90%	95%
0	2	5	7
1	4	8	10
2	6	12	14

Where those numbers likewise have been demonstrated (at least in experience, if not in actuarial tables) to be a little over-optimistic, but even then, they're "not great" to start with. But they certainly are a better starting point for someone trying to do genealogy rather than simply testing for ancient ancestry, general population studies, or very basic paternal line testing.

I don't always need a Y25, Y37, or Y67 test to determine someone "probably isn't" descended from a recent ancestor(so long as I'm able the view the full STR result panel, and not the filtered ones FTDNA gives in the matching tool). A Y-12 test is generally capable of that, if they're more than a GD2 match, the odds are that "they most certainly are not."

What you need a Y67/Y111 test for is to strongly support (using STRs) the idea that a particular male ancestral line is in fact common, because even very distant relations can share STR markers with you. Which is why SNP testing is better still.

For family line testing, you use Y12, Y25, and Y37 to rule matches "out"(but only if you can get their STR values and verify the match is even more distant than FTDNA's criteria allows for), you use Y67, Y111, and SNP/BigY testing to rule relationships "in."

[Jim Barrett]

We differ then.

If you had bothered to click on the link above, and examine the spreadsheet of 12 STR matches, you would see something that might surprise you. If 80% of "matches" are in different haplogroups, it's not a match list, it's a fantasy list -- because if haplogroups don't match, it's misleading to instruct customers to pursue genealogical leads wholesale based on that list of "matches."

It's irresponsible. Should be corrected.

Your link does not show the STR values of your Y-DNA12 matches. It isn't FTDNA's fault you don't understand what they are comparing to obtain the list of your 12 marker matches. Nothing irresponsible. Nothing for them to correct.