Announcement

Collapse
No announcement yet.

The Big-Y matching program is robbing you of your correct matches.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • clintonslayton76
    replied
    One of my haplo admins described Big Y as "flawed" and encouraged me to use a comprehensive Y chromosome sequencer, but beyond the issue that the recommended service refused to service my order, the fact that I already had three non-cooperating DNA labs servicing my requests meant that I had hours of homemade spreadsheet analysis instead of genealogical research time, in a field where I am by no means expert and prone to overlook a mistaken value where such errors are not obvious.
    I understand the counter-arguments but I agree with the original complaint. It seems impossible that I "suddenly" had five levels of SNP hierarchy from I-Z2535 down with zero matches as of the introduction of Y500.
    Last edited by clintonslayton76; 3 January 2019, 12:14 PM. Reason: left out word

    Leave a comment:


  • wkauffman
    replied
    Originally posted by Ericjle View Post
    Mudgeeclarke: The reason is quite simple. FTDNA needs to rewrite their Big-Y matching program. As an example, we had a new Big-Y results come in, NO Big-Y matches, when they do. One of these two had 23 Not-Named (Novel Variants) plus there are 6 SNPs that define the sub-clade, so 29 total. One less than FTDNA's limit of 30. So FTDNA must be using bogus or unreliable SNPs in their matching program. This problem goes all the way back to when FTDNA first started to offer the Big-Y test. I have emailed FTDNA multiple time about this problem.

    You really don't seem to get it when it comes to the RELEVANCE of the matches being presented. When counting to 30 you count for BOTH individuals not just the one back to a haplogroup level!!! So 23+xxx >30 and you won't see them. The current limit takes us back to about 2000 years ago. That is a good starting point at which STR motifs begin to solidify and can be used to identify other non-BigY result which may bring the haplogroup level closer to current time.

    Referring to "bogus or unreliable SNPs" in relation to the other results needs more clarification. FTDNA's matching system is NOT, like Iain McDonald's (R-U106), account for where the appearance/recurrence is established based upon a branch and group of results. FTDNA is a straight comparison against what they feel is valid. Now FTDNA IS incorrectly filtering out valid calls, and some corresponding haplogroup levels, based upon their localized way of establishing the validity of a call. They seem to be filtering out entire segments without properly processing whether some of the calls are reliable due to the recLOH. Over time individuals ARE losing SNPs from their results based upon FTDNA's filtering mechanism which are relevant to that specific result.

    Leave a comment:


  • bartarl260
    replied
    Originally posted by Ericjle View Post
    Mudgeeclarke: The reason is quite simple. FTDNA needs to rewrite their Big-Y matching program. As an example, we had a new Big-Y results come in, NO Big-Y matches, when they do. One of these two had 23 Not-Named (Novel Variants) plus there are 6 SNPs that define the sub-clade, so 29 total. One less than FTDNA's limit of 30. So FTDNA must be using bogus or unreliable SNPs in their matching program. This problem goes all the way back to when FTDNA first started to offer the Big-Y test. I have emailed FTDNA multiple time about this problem.
    I think you misunderstood an important part here:

    The total number of SNP differences between a match can only be 30.

    So if a person tests and comes back with 29 Novel SNPs, in order to match, the other person would only be allowed 1 additional solitary differing SNP value. Although they should share the 6 SNPs that define the clade, so the real total variation "allowed" for them in such a case would be 7 novel SNPs, compared to 23 for your kit. Not likely in either case unless your respective branches are extreme outliers in SNP mutation rates, just in differing directions(highly stable vs highly unstable).

    As it is, with my results back, the discussion here is largely moot for me, I have 44 people who match closely enough for me to see them through BigY, and they're all within two steps of my currently assigned terminal SNP so my tree can branch up to 3 more time before I lose any of them.

    I just have the current mystery of how the 2 kits on my terminal SNP aren't STR matches(although with 19 and 20 differing SNPs respectively 8&9 of which are named SNPs, I could guess as to how; interestingly, I also share a unique SNP with each match that isn't shared with the other but Biology is often messy), while most of the other 42 BigY Matches are. Although the closest of those 42 matches as per TiP don't cross the 50% point(@111 markers) until 18 to 20 generations back, so still not likely to be close enough to prove "Genealogically useful" particularly given none of the surnames match and 5 generations takes me back to 1800 on that line.

    Edit: And I just checked to be sure, my most distant matches have 30 non-matching SNPs(with me, not my Terminal), so that does seem to be the cutoff.
    Last edited by bartarl260; 2 January 2019, 12:17 PM.

    Leave a comment:


  • Ericjle
    replied
    Mudgeeclarke: The reason is quite simple. FTDNA needs to rewrite their Big-Y matching program. As an example, we had a new Big-Y results come in, NO Big-Y matches, when they do. One of these two had 23 Not-Named (Novel Variants) plus there are 6 SNPs that define the sub-clade, so 29 total. One less than FTDNA's limit of 30. So FTDNA must be using bogus or unreliable SNPs in their matching program. This problem goes all the way back to when FTDNA first started to offer the Big-Y test. I have emailed FTDNA multiple time about this problem.

    Leave a comment:


  • Mudgeeclarke
    replied
    I am FTDNA 206005 Clarke and my closest match is 392206 Deed. We have done Big Y. We are both at Y Full, and on The Big Tree. We are S-1491 > R-Y23824 on the YFull pages, and R-8493901-C-T at the Big Tree. At Y Full we have 11 common SNPs; the next closest to us at S1491 only have 2.

    He and I correspond from time to time because we have a similar problem - parental G GFs were NPE. And we “think” our parental interests are only one County apart in the UK. However, the timeframe to MCRA is said at Y Full to be over 1700 years.

    The Problem. What neither of us understand is how at FTDNA we don’t show up as a match to each other at any level at any page.

    Can someone please explain how this can be so. Is there any test we can do to close the time gap.?

    Leave a comment:


  • clintonslayton76
    replied
    Thank you, I had done all of the tree examination you mention at both sites.

    >> I recruited two distant cousins to test Big-Y and went from 2800 ybp to 175 ybp.

    That is why I recruited a cousin this month, in the same hope.

    I note here that YFull regularly informs me of any changes and updates to their tree, and that FTDNA most emphatically does not. As stated, my terminal at FTDNA has changed three times without any hints from FTDNA, I just happened to notice them because I look at my project so often. I find that very strange that I have to explicitly turn off 12-panel STR matches, but do not get any notifications of a fundamental change arising from the most expensive FTDNA test I have ever taken. (I was denied service from another, even more expensive and complete Y sequencer because I requested an estimated date of results, and my over $700 check was returned, that is another story in another thread.)

    I am in all of the haplo projects for my I group, check them regularly, and am aware of and have contacted the specific admin on I1, from whom I have recorded some comments. My grouping at the I1 link you provide is with a grouping which does not include some of my matching YSTR members who have not done any test that would confirm equivalent A1892, even though I would predict that they would. Naturally I would have put them in the same group as I, but I understand that the lack of confirming SNP as opposed to predicted SNP is the dividing factor.

    I am not happy that the access to projects of members from their kit pages is not available to admins any more during "impersonation." Lots of my members did not understand that even ADVANCED access is not truly FULL access, and so I cannot even look at my own personal project results from my own kit page when I am signed in as admin, which I am for about 90% of the time. This is extremely annoying when my members are asking me about their presence on these sites, since most of the time they cannot find themselves. I am a member of one of three haplogroups in my surname project, and strongly encouraged all members to join their respective haplo projects..

    Ken Nortdvedt stated to me that he no longer accesses the page because he does not have the bandwidth, (Indeed, I get "Houston we have a problem" more than 50% of the time when I try to see my results there and am getting that right now.) It is Ken that has "named" my haplo as Anglo Saxon 889. I have his modality spreadsheet to look for my matches since FTDNA has made it just about impossible. About the only project where I have seen enough data to "bridge" the time-scales between SNP and STR testing is Maurice Gleeson's and he is clearly better-funded than I; we have only 3 men of our 50 members who have done Big Y and I just cannot fund these myself, even at sales prices.

    Again, thanks for spending the time, basically duplicating my own efforts, but it is much appreciated. I still agree with the spirit of your original complaint, but I understand that by the lights of FTNDA's SNP interpretation, I have no matches. I have a few months to wait to see what happens with my cousin.
    Last edited by clintonslayton76; 28 December 2018, 11:34 AM.

    Leave a comment:


  • Ericjle
    replied
    clintonslayton76 Your statement “but have been assured by every popular treatment on the subject (Bettinger, Dowell) that the more men do the sequencing, the more likely are historical-time SNPs to be discovered” This is absolutely true. As an example, I recruited two distant cousins to test Big-Y and went from 2800 ybp to 175 ybp. This created a new sub-clade, that found two more Big-Y matches that created two more sub-clades above me.

    YFull has you
    -A1890A7425 * A1890 * A1892 formed 3000 ybp, TMRCA 2900 ybp with no sub-clades below,that match with Sweden
    There is a difference between YFull and FTDNA is because the other persons probably didn’t join or there is a difference between reliability of the SNPs between the two companies.

    At 2900 ybp you probably have a lot of not-named SNPs, anything over 10 will go over FTDNAs matching limit of combining all SNPs limit of 30.

    I-Z2536 has 3 other brother sub-clades below, you will not match any of them

    Go to www.familytreedna.com/public/y-dna-haplotree/A This phylogenetic tree is not just Big-Y, it includes other SNP tests

    Enter i-F1997 into “go to branch name”

    The first version of this phylogenetic tree had persons with no country entered listed, but the 2nd version deleted them. FTDNA shot themselves in the foot again. Now you have to go to the 3 vertical dots, which is confusing.

    Click on the right 3 vertical dots for matches

    You have 4 no country entered matches

    At FTDNA the Sweden match is a brother sub-clade to you, so a different sub-clade

    Next go to I project http://www.familytreedna.com/groups/...out/background

    Its brick walled for non-members, if you are not a member please join. You will most likely find you matches there.

    There is another way to find Big-Y matches by looking at your y-12, y-25, y-37 match list by clicking on the “Only show Big-Y”

    Leave a comment:


  • clintonslayton76
    replied
    Ericjle, If I understand your question correctly: FTDNA gives me the path: I-Z2535 >CTS10937 >Y3153 >A1890 >F1997, and YFull gives me I-A1890 as terminal. I used to have matches before FTDNA went down this path. As stated by others, they were not particularly meaningful, because FTDNA gives no YBP values, but I was down to a 2300 YBP from YFull/ISOGG, and hoping to see some refinement instead of losing all contact with even distant men. I was fortunate to retain the contact information for a Swedish gentleman who has been a lively correspondent, common ancestor within 20 centuries or not.

    Any help would be appreciated, I am writing a book about the fundamental surname findings for more than three lines, and writing (and self-editing, a foolish enterprise) takes most of my energy these days. It is not clear to me how you would be able to do this, since my haplo admin states that FTDNA's newest terminals subvert the process.

    Leave a comment:


  • John McCoy
    replied
    "Robbed" seems too strong a word, but it is clear that FTDNA could do a much better job of putting the Big Y results into a meaningful context. As far as I know, my best matches are still likely a few hundred years in the past, so they are nothing to get excited about from a genealogical viewpoint. They do, however, demonstrate clearly that my McCoy family is not connected with any other McCoy family that has been tested, and that is a very useful piece of information from Big Y. But I obtain that context from Y Full and from the Big Tree, not from FTDNA.

    The desired context should not be constrained by a fixed number of SNP's, but rather, and if at all, by some number of matches, displayed in such a way that it is clear what sort of distances separate them based on their respective locations in the haplotree, with also an option to display STR data and STR distances. In other words, the display I want to see should answer the question, "what does my haplotree neighborhood look like, and who are my neighbors?" The Big Tree does most of that very well, by the way!

    Leave a comment:


  • Ericjle
    replied
    clintonslayton76: What is your terminal sub-clade name? I may be able to find your Big-Y matches, by going to FTDNA projects or better by going to the new phylogenetic tree.

    Leave a comment:


  • Ericjle
    replied
    bartarl260 The 30 SNP differences includes all the SNP's that define the sub-clade plus all the Not Named (Novel Variants) plus unreliable SNPs. I proved this to FTDNA, they replied to me for comments to the IT team. Many people have over 30 Not Named (Novel Variants), so they will never have Big-Y matches.

    Leave a comment:


  • bartarl260
    replied
    Originally posted by Ericjle View Post
    The posts above made some good points, but what I am referring to is people who took the Big-Y test and have NO matches, when they do, even their exact terminal sub-clade matches. It is impossible to contact them to find out if they are related. I know of over 10 known exact terminal sub-clade matches who have not joined any projects. So you can not find them that way. They don't know to join, because they have no matches.
    Until there is evidence to support a higher than currently expected mutation rate in Y-DNA SNP's, if you're more than 30 SNP mutations removed from your terminal SNP Match(which I understand is the cutoff at present for BigY), you're probably not related in the past 500 years at the earliest.

    Even going with 5 generations/century and 1 mutation per generation(and no regressions), for both sides of the match. Your earliest connection would be 300 years before the Y-DNA tester was born. But IIRC, ISOGG is seeing indications that typical average generation for men is closer to 33 years, not 20. 15*33=495 years and considering that the average Y-DNA mutation rate is expected to be a little bit slower than that...

    Leave a comment:


  • clintonslayton76
    replied
    As I perhaps did not make clear, I have 15 Y STR matches to paper-trail males and NO Big Y matches with anyone. That is why I will be looking at a cousin's Big Y500 results, to test your theory. If FTDNA's Big Y matching criteria is too specific, it should be obvious when patrilineal KNOWN relatives are tested.

    I was advised by three haplo admins of three different projects (and repeatedly), that spending $ on testing my Y STR matches for Big Y was a waste, because they would by definition still match their Y STR cousins. They also responded to individual queries from members the same way: they said only ONE member each of my largest groupings (I-M253 and I-P37.2) needed to test SNPs because cousins would learn nothing new from the expense.

    Yet another admin advises men to test only 12 STRs and then spend their money on individual SNPs, which I find to be hare-brained from the point of view of a genealogical surname project, but sensible if the only goal is to save money and get the most general haplogrooup assignment. As stated by bartari260, our goal is to plausibly corral the patrilineal lines found in documentation, not to find out which continent our ancestors lived on 30,000 years ago. I am aware that those time-scales are more frequently seen from SNP testing, but have been assured by every popular treatment on the subject (Bettinger, Dowell) that the more men do the sequencing, the more likely are historical-time SNPs to be discovered. I hope they are correct. But as I infer from the original complaint, explaining this to potential samplers when FTDNA comes back with very specific terminals that make for non-matches makes it a hard sell.

    Leave a comment:


  • Ericjle
    replied
    The posts above made some good points, but what I am referring to is people who took the Big-Y test and have NO matches, when they do, even their exact terminal sub-clade matches. It is impossible to contact them to find out if they are related. I know of over 10 known exact terminal sub-clade matches who have not joined any projects. So you can not find them that way. They don't know to join, because they have no matches.

    As far as the Haplogroup Admin., most of the Haplogroup Projects are so large that its impossible for the Admin. to individually help every member with their goals. It took me 3 months of appeals to get my small Haplogroup project started. Now with 100 members we are now down to genealogical timeframes, 7 branches are down to 50 to 300 years ago with Big-Y. We have solved several adoption and NPE cases.

    Leave a comment:


  • clintonslayton76
    replied
    I feel the pain of the Ericjle, and glad I saw his post. While another service (YFull) gives me a terminal that matches me with someone who is not a Y match and is many thousands of years back for common, there has to be a middle ground. FTNDA have altered my terminal and my wife's uncles terminal to what appear to be very novel or private values. It would be great to know if anyone else has them, or if they came up within 300 years, but by the same token, where are the age estimations or explanations for any of the SNPs assigned by FTDNA? I am in the dark about values that are not recognized on ISOGG, especially if they are prefixed BY.
    I learned today that after goading by me, a known cousin of mine took advantage of the sale and is the second of my "line" (after me) to do Big Y, he comes down from a man who is probably the brother of my ancestor from flourishing in 1771 Surry Co North Carolina. We have very strong paper-trail genealogy and his GD is 1 from me with 111 STRs, and our common ancestor is at least 6 gens back, So in March or so, I will see what he shows for Big Y-500 STR Differences and matches. I am also encouraging him (when he gets results) to upload to YFull, because I think everyone should have an alternate interpretation of something that is this expensive and (typically) far removed from the Y STR tools and grid. By my estimation, he would have to be a match, as I used to have when my terminal was I-A1890. I find it very difficult to promote this test among my 50 members because of what Ericjle is writing about, the almost total disconnect between Y STR and SNP testing. I know why it is there, but there seems to be little effort on the part of haplo admins to bridge the gap between their goals and mine, which, as bartari206 writes, is to see meaningful groupings to support genealogy, not with some SNP match that serves to disconnect me from people that I know are my patrilineal cousins.
    Last edited by clintonslayton76; 23 December 2018, 05:04 PM. Reason: spelling corrections

    Leave a comment:

Working...
X