Announcement

Collapse
No announcement yet.

Bug when downloading concatenated Family Finder raw data

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Bug when downloading concatenated Family Finder raw data

    On Nov-16 I've got results for my Family Finder test.
    I can see this results on "Matches" page and all other tools that FTDNA provide.
    But on the "Download raw data" page there is some trouble.
    First of all for several hours any button on this page yielded something like a page with 404 error ("Houghston we have a problem") or a page with some xml-text.
    After about 8 hours downloadable files appeared under buttons. But "concatenated" files for any version (37 or 36) looked quite strange.
    When "autosomal" or "X" gzipped partial files downloaded normally and contained valid csv tables, "concatenated" gz-files contained only the table with a part of Chromosome 1 information. It is interesting to notice that the size of this table is exactly the same as of "X" csv table (true for both 36 or 37 versions).

    My question is: does anybody meet such bug in recent days?


    Of course, I've sent the request to support team of FTDNA, but there is 3rd trouble: since Nov-17 I have no reply from them and do not see any activity in solving this problem.
    Attached Files

  • #2
    I hope they will fix the glitch. They should.

    However, the "raw data concatenated" file has the same information, as the other two files you had already downloaded. That is after the autosomal results, the X chromosome results follow in the same file.

    You are not missing anything. If you need or want the concatenated results, you can create them yourself. Doing it on Microsoft Windows in a command prompt window is easy, but Notepad can that too. The command line version is like below (names of your files would incorporate you kit id, not 7xxxxx)
    Code:
    C:\Users\You\Downloads> copy /B 7xxxxx_Autosomal_o37_Results_20171123.csv + 7xxxxx_XChromosome_o37_Results_20171123.csv concatenated.csv
    7xxxxx_Autosomal_o37_Results_20171123.csv
    7xxxxx_XChromosome_o37_Results_20171123.csv
            1 file(s) copied.
    
    C:\Users\You\Downloads>

    Mr. W

    P.S.
    The result of the above command would give you the result identical to the file normally provided by FTDNA.
    Last edited by dna; 23rd November 2017, 01:54 PM.

    Comment


    • #3
      It is interesting to understand is this fault single (e.g. due to damaged record in data base) or common (e.g. when some script takes 2 records from data base and gzip them into one file).
      Not every customer can notice this problem and not everybody is ready to do FTDNA's work with the help of command prompt.

      Comment


      • #4
        Originally posted by kom View Post
        It is interesting to understand is this fault single (e.g. due to damaged record in data base) or common (e.g. when some script takes 2 records from data base and gzip them into one file).
        I am guessing the files are kept generated by FTDNA, so we would need to ask those who had their results in the last 7-10 days. I do not have time today to troubleshoot, but if you could please take a look at the end of your Autosomal Results CSV results file, does it look OK? Anyway, FTDNA does a very good job with genetics testing, IT and web frontends are not their forte...

        Originally posted by kom View Post
        [----] Not every customer can notice this problem and not everybody is ready to do FTDNA's work with the help of command prompt.
        Well..., I think the average customer does not download those files. And I understand that many (most?) of those who download those files do not look at them. They only upload them somewhere else to share. I do not use GEDmatch, but they used to require separate uploads for autosomal and X chromosome. Clearly there is some need since FTDNA is providing concatenated files. May be you know where or why concatenated files are useful?


        Mr. W

        Comment


        • #5
          Originally posted by dna View Post
          I do not use GEDmatch, but they used to require separate uploads for autosomal and X chromosome. Clearly there is some need since FTDNA is providing concatenated files. May be you know where or why concatenated files are useful?
          The last reqirement of GEDmatch is single concatenated file, not separate uploads. And if you try to upload the file without X part you will receive something like "file have not been tokenized".

          Comment


          • #6
            Originally posted by dna View Post
            I do not have time today to troubleshoot, but if you could please take a look at the end of your Autosomal Results CSV results file, does it look OK?
            This file looks normal and ends with rs3888396 on Chr. 22.

            By the way, if you concatenate two files with command prompt you have to cut 1st row (header) of the 2nd file with some text editor.

            Comment


            • #7
              Some concatenated files have been missing the X Data
              It requires notifying FTDNA to fix

              Concatenated should end with X, not 22

              or when I run into it, I just download the same build of X Data.
              Open both in Programmers Notepad (free program), Hit Ctrl A in the X data one.
              Go to bottom of Autosomal one a on line after the last chromosome 22 entry, hit Ctrl V.

              Save as csv file, and upload to Gedmatch
              I have had no issue uploading unzipped files.
              to zip as a .gz file use 7zip.

              Normal concatenated files have X Data header in file. (not needed, but it is there)
              link to image on onedrive
              https://1drv.ms/i/s!Al27wnXopRKxhz5C5r_wsIoKDRIO
              Attached Files
              Last edited by prairielad; 24th November 2017, 01:51 AM.

              Comment


              • #8
                Autosomal file ends with Chr. 22, not concatenated. This broken concatenated file ends with part of Chr.1, see screenshot above.

                Comment


                • #9
                  Originally posted by prairielad View Post
                  It requires notifying FTDNA to fix
                  As I have mentioned earlier I have sent request to FTDNA on 11/17/2017.
                  Unfortunately there is no any traces of activity from support.

                  Comment


                  • #10
                    I have found that at least one file in top of my match list on Gedmatch site (uploaded recently, green in this list) has the same ('has not been tokenized') problem. It is quite strange fact that it has appeared in one-to-many comparison, but it has.
                    So I think that a) it is the very common problem and b) it is difficult to notice it for common user. These damaged files will appear on GEDmatch if FTDNA will not repair this bug.
                    Last edited by kom; 25th November 2017, 03:20 AM.

                    Comment


                    • #11
                      Uh oh...
                      First I didn't find similar problems using forum search form.
                      Now vBulletin shows topics that tell that it is the bug since 2015 at least.
                      Hougston problem pages, xml-pages and then - files without part of information.
                      Nothing has been done.
                      FTDNA, you are the best!

                      Comment


                      • #12
                        Thank you. I finally found what's wrong with dad's file and fixed it. It's all working now.

                        Comment

                        Working...
                        X