Announcement

Collapse
No announcement yet.

Recent changes to old Big Y results

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Recent changes to old Big Y results

    I have seen several comments from Project leaders on chatgroups / forums that there has been an overnight massive change in people with known Big Y's data - one person has had his novel variants drop from 240 to 82, another from 220 to 59. There are some very frustrated and cranky group leaders and researchers out there and one of the biggest gripes is a lack of answers as to how and why this has happened. These people had spent hours/days/weeks working through massive spreadsheets and overnight things had disappeared or was re-named. Some very senior researchers are threatening to walk away from the projects which would be a tragedy for those of us who struggle to understand the science. Can anyone respond in this public forum ?

  • #2
    The situation is more perplexing than that.

    One project member received his Big Y results on April 30, classifying him as Q-Z780 (or Q-CTS1780, which is equivalent). The results looked complete and self-consistent, and also consistent with his 25 STRs. They were not consistent with his alleged patrilineage, but he was adopted and himself suspected some unreliability in the ancestry listed on the adoption papers.

    Today, FTDNA re-released his Big Y results. He has been reclassified as R-Z367, but more importantly: His Big Y results look incomplete and inconsistent, rather like gibberish.

    I sent email to Janine Cloud, but she is out of the office due to a death in the family.
    Last edited by lgmayka; 29 May 2014, 06:16 PM.

    Comment


    • #3
      In another project I co-administer, 4 Big Y results were re-released today.

      One was previously CTS1211+ and CTS3402+ , as expected from his 67 STRs. Now those are gone, and he is classified only as R-Z280.

      Comment


      • #4
        Obvious.....

        To be frank it's rather obvious. FTDNA used a subset of the known SNPs in their initial analysis. They did not utilize the ISOGG SNP compendium and additional 1Kg mined SNPs nor those from Full Genomes that had been made public as the basis of the initial comparison. There are now about probably >25,000 additional named SNPs known than what they used. So if FTDNA goes through a reanalysis now a bunch of novels will show up as named. A second topic is that there is a % of the 'novel' SNPs which are inconsistent or invalid calls even though they came out of the caller with high quality scores.

        If one takes a look at the Big-Y results that have been processed by FGC you will get a better understanding of what FTDNA novels disappeared. In the U106 project we have had the services of a great programmer who has created a system that has done a reasonably accurate parsing of the Big-Y VCF file to remove the known and inconsistently called SNPs. Our filtered results are very close to what FGC or YFull analysis filters out as high quality calls.


        We have a very solid draft of the haplotree in our region due to the quality of the filtering and comparisons that we have been able to efficiently perform. See https://app.box.com/s/afqsrrnvv2d51msqcz2o Note that some occurrences of recurrent SNPs are excluded by the way the analysis was performed. Some recurrent SNPs will have genealogical meaning for some of the lineages.

        Comment


        • #5
          Originally posted by lgmayka View Post
          The situation is more perplexing than that.

          {snip}
          Today, FTDNA re-released his Big Y results. He has been reclassified as R-Z367, but more importantly: His Big Y results look incomplete and inconsistent, rather like gibberish.
          It has been noticed that when FTDNA does a rerelease of an individual's Big-Y results there are problems with the files. Until FTDNA announces a reanalysis of the original results I would be leary of any republishing of the original results.

          Comment


          • #6
            And my husband's results are now missing entirely, with a note that results are expected in one to two weeks. It could be due to Wayne's suggestion about reclassifying novel variants.

            Comment


            • #7
              Originally posted by lgmayka View Post
              Today, FTDNA re-released his Big Y results. He has been reclassified as R-Z367, but more importantly: His Big Y results look incomplete and inconsistent, rather like gibberish.
              Can you send me the affected kit numbers for this one and the others by private message? I'm checking into this.

              Thanks,
              Elise

              Comment


              • #8
                Originally posted by Ann Turner View Post
                And my husband's results are now missing entirely, with a note that results are expected in one to two weeks. It could be due to Wayne's suggestion about reclassifying novel variants.
                Can you send me your husband's kit number?

                Thanks,
                Elise

                Comment


                • #9
                  Missing Big Y Results

                  My dad's Big Y results are missing too. His page now says "Awaiting Results"--in 1-2 weeks. But under "Order History" it says completed 5/29/2014. They were completed three weeks ago.

                  Has FTDNA sent out any communication regarding this situation?

                  Comment


                  • #10
                    The questions are

                    1) have the bam files stayed the same? (If not ...
                    it was a sample mixup, but all the bams I have looked at
                    clearly were perfectly in agreement with what I expected, including all SNPs and spotchecked
                    STRs, of which of course there are not that many.)

                    2) Are the beds and vcfs the same? I must admit that I have not looked at many web pages, just the vcf/bed and bam files. The beds/vcfs agrees with the bams, except in rare cases where examination of the bam showed that the caller
                    program was apparently expecting a diploid organism.
                    At Clan Donald we only had one of these changes, and I do not have the old bed, vcf, or bam.

                    3) What's going on?

                    Comment


                    • #11
                      While I am absolutely baffled that FTDNA would have taken such action without first notifying its Big-Y customers and providing and some form of an explanation, I am relieved that no changes have been made (so far as I can see) to my Big-Y results.

                      Would these changes also impact any BAM file analyses performed by YFull?

                      What has been brought to light here certain does not demonstrate any claimed movement towards improving customer communication...

                      Comment


                      • #12
                        Having reviewed one member's results that were changed last night I can guarantee FTDNA aren't just removing upstream SNPs, several pertinent to his cluster are also missing.

                        I'll have to have a look at his VCF and BED files to get a better idea of what's going on but of course that's not so easy since FTDNA's bonkers decision to limit access to these, when is that peace of nonsense going to be rescinded ???

                        If this information has been stripped from his VCF file I'll be extremely perturbed !!!!

                        Comment


                        • #13
                          We have to assume that someone at FTDNA explained to someone higher up why some Big Y results had to be reissued. At that point, an opportunity was missed! That's when a customer communication should have been crafted and approved. The explanation could be benign -- "SNP's have been reclassified as a result of progress, etc." -- or there could be other explanations. Whatever the cause, the explanation should show up on the customer's account page, as a "Message" (that's what the messages section is for), or as a banner or link in the Big Y results.

                          Another question needs to be answered by FTDNA: Was this an isolated occurrence affecting just a small number of Big Y kits, or are there more kits in the pipeline whose changes haven't yet been posted?

                          Comment


                          • #14
                            Feeling a lot better about this after having checked the above mentioned participants results again, four of the five cluster defining SNPs have returned plus another a little further upstream.

                            This looks like a work in progress, are FTDNA actually trying to refine the haplotree via Big Y results ?

                            Comment


                            • #15
                              Originally posted by lgmayka View Post
                              Today, FTDNA re-released his Big Y results. He has been reclassified as R-Z367, but more importantly: His Big Y results look incomplete and inconsistent, rather like gibberish.
                              This problem, and the other one I mentioned, appear to have been remedied--perhaps simply by reverting to the earlier released results.

                              Comment

                              Working...
                              X