Go Back   Family Tree DNA Forums > Family Tree DNA Communications > Features Requests & Bug Reports Area

Features Requests & Bug Reports Area This forum is for customers to talk about features they would like to see. It may also be used to report potential website issues and account bugs, though you may get faster service by sending those through the Open a Request feature found in the Customer Service link at the top of the forum page.

Reply
 
Thread Tools Display Modes
  #161  
Old 12th October 2017, 03:14 PM
chrisbonisa chrisbonisa is offline
FTDNA Customer
 
Join Date: Oct 2017
Posts: 7
Quote:
Originally Posted by ltd-jean-pull View Post
This post dated the 27th August was the first to give us a hint about what might be causing this current issue with uploading Ancestry autosomal data to FamilyTreeDNA being due to a change in file size -i.e. Missing data.

Some might think that FamilyTreeDNA ought to make it a priority to sort out this issue so that they can upload a file that it is a different format from the the files FtDNA is set up to accept, even though FtDNA doesn't get informed by Ancestry that they're going to change the files. Ancestry can't inform their customer service reps or customers of the file change so I doubt they'd pick up the phone and tell a competitor.

Anyway, back to August 27th. What a shame that some computer geek who works for FtDNA who was stuck at home with water lapping around the porch didn't somehow manage to log-in from home and start working on the coding. And how unreasonable of them to close down the server for a few days meaning that e-mails couldn't be received during the flooding.

I can understand people's frustration, but some people need to get some perspective. FamilyTreeDNA's priority is their PAYING customers who tested with them, and in the last few weeks they've been working in very trying circumstances.

Hopefully it will be sorted in due course, but it wouldn't surprise me if Ancestry continues to muck around with file size. I have no idea why they do it.
This is completely the wrong approach. First of all there are TONS of us who do the transfer and pay to upgrade it to get the chromosome browser. As a matter of fact, FTDNA isn't of much use to me without that as they have far fewer people than Ancestry or 23andme. FTDNA can only be successful if they allow transfers to build up their user base over time. Also, lots of us, me included, pay for other products and want to have all of our data aggregated in one place. I paid the $160 for y-dna-37 test and will most likely upgrade that(+$100). And the $19 for each transfer upgrade is pure profit for them and likely the same amount of profit they receive from autosomal tests directly through them after lab costs.

Regarding the hurricane, I run an IT company with a datacenter and central office in Tampa, FL.....you know, where hurricane Irma hit. We had emergency contingency plans in place for this sort of thing years ago and because of that, the impact was nill even though power was out for almost all of our local employees and flooding everywhere. We are no where as large as FTDNA and they are more than capable of handling emergencies, and I'm sure they did in fact deal with it in a responsible way. I don't think this problem is in any way related to the hurricane. Not to mention that was like 6 weeks ago.
Reply With Quote
  #162  
Old 13th October 2017, 09:25 AM
aprilmcg123 aprilmcg123 is offline
FTDNA Customer
 
Join Date: Sep 2017
Posts: 9
Quote:
Originally Posted by chrisbonisa View Post
I've just done a similar comparison using php/mysql database and came up with the exact same results as you with my broken file(2.0b):

v2.0a_count - 668942
v2.0b_count - 650410
diff_count - 18532
v2.0a_missing - 1354
v2.0b_missing - 19886

This suggests its not an error on the part of Ancestry but either a different chip or simply different parsed results for some reason. This also means its most likely FTDNA just not able to handle this new scheme, its probably seeing the "2.0" in the file header and looking for some exact snp schema in the file and since it doesn't match it rejects it. Not counting the new unique snps, this is merely a difference of ~3% which should still be plenty of overlap for FTDNA to handle this, if they would simply try. All other sites are able to accept this type of export without a problem and generate matches from it, I've tried gedmatch, myheritage, wegene, dnaland, etc. FTDNA is the only one erring out...

FTDNA - you are losing money and customers, Ancestry is utilizing this new schema for more and more exports, we need a fix ASAP!
Unfortunately, this doesn't make sense as being just a new chip issue between Ancestry and FTDNA as I have evidence of several profiles processed using the same chip (V2 according to the file) within days of each other where the size difference is occuring and some of them will upload and others (those with the missing SNPs) will not.
Reply With Quote
  #163  
Old 13th October 2017, 02:58 PM
chrisbonisa chrisbonisa is offline
FTDNA Customer
 
Join Date: Oct 2017
Posts: 7
Quote:
Originally Posted by aprilmcg123 View Post
Unfortunately, this doesn't make sense as being just a new chip issue between Ancestry and FTDNA as I have evidence of several profiles processed using the same chip (V2 according to the file) within days of each other where the size difference is occuring and some of them will upload and others (those with the missing SNPs) will not.
I'm curious about this myself, I wonder if its a new chip not fully rolled out yet? Or perhaps new software? But yes I agree, its inconsistent and confusing.
Reply With Quote
  #164  
Old 15th October 2017, 11:13 AM
WilliamKF WilliamKF is offline
FTDNA Customer
 
Join Date: Jul 2015
Posts: 2
GEDmatch works fine, but FTDNA rejects

I'm having this issue too with my cousin's results that just posted at Ancestry.com last week. GEDmatch.com accepted the transfer with no issues, same for MyHerritage.com.
Reply With Quote
  #165  
Old 17th October 2017, 09:51 AM
Zack Zack is offline
FTDNA Customer
 
Join Date: Jul 2016
Posts: 4
AncestryDNA kit received on 10/17/17

Upload attempt (many) on 10/17/17

Can not upload successfully, headers were already the values suggested earlier in the thread.

Error Code: 'The file is an unsupported version or in a corrupt/malformed format.'

Zip file is 5,607 KB
Extracted *.txt is 17,184 KB
Reply With Quote
  #166  
Old 20th October 2017, 12:56 PM
hansonrf hansonrf is offline
FTDNA Customer
 
Join Date: Sep 2012
Location: Pennsylvania, USA
Posts: 340
Same

Any response yet from FTDNA?
Reply With Quote
  #167  
Old 20th October 2017, 06:29 PM
OldFinneyKid OldFinneyKid is offline
FTDNA Customer
 
Join Date: Sep 2016
Location: California
Posts: 22
Question They could fix this

I noticed the file size difference in August but did nothing else since my files were not affected. This week I had a file that was the small size and could not be fixed with the header change. It took me nearly a day to write the program to fix it, and I'm not a professional programmer. Here is what I discovered (since I put some counters in my program):

The working V2 Ancestry file has 668,962 lines, 19 of which are header and the last one is blank.

The broken Ancestry file has 650,430 lines, with the same header lines and final blank line.

Both files have a number of "null" records where the allele values are "0" rather than A, T, G, or C. Ancestry documents this. There are also some records with values of "I" or "D" -- I can't find documentation of these (ideas?).

The rs identification strings are different for some of the locations; however, the positions on the chromosome are consistent.

Given all these factors, I was able to create a master template with all null values (leaving the I's and D's where I found them, for no reason). I then matched up the chromosomes and locations, giving every location the same rsID name as the known good file. I then inserted the results (A,T,C, or G) from the damaged file.

During the comparison, there were 1,048 records discarded without a matching rsID from the good V2 file.

The resulting "fixed" file is the exact size and length required by FTDNA upload and includes results ONLY from the DNA subject's faulty file. Missing data that was replaced is null as described by Ancestry's documentation.

To test the result, I loaded the original Ancestry raw download and the "fixed" file onto GEDmatch. I then ran a one-to-one compare. The match is identical, similar to running the kit against its own kit number. The new results are only 2-10 SNPs different from the original (expected with the missing data). I have tested this on 3 of the damaged (size of 17,184 KB) files with consistent results.

GEDmatch reports all cM matches between the "before" and "after" files are exact, all matching segments are the same length, with the same start and end positions. This new file does upload to FTDNA with no complaint.

During the conversion, several details were noted:

Chr 1: 50618 records added (52 were already null) -- 63 records were discarded // 27 modified rsID's

Chr 2: 52461 records added (64 were already null) -- 49 records were discarded // 25 modified rsID's

Chr 3: 41069 records added (45 were already null) -- 41 records were discarded // 11 modified rsID's
.....

I could go on, but those of you following this thread get the picture: It took me just over a day from decision to completion, and it's not my business.

I would share the program (it runs in a command shell on Windows) but I'm not set up for testing on different computers and not sure I want the liability for user error. But we're talking about less than 500 lines of ugly, amateur C++ code that I'm too self-conscious to post publicly. But if I can do it, Ancestry could easily provide this and FTDNA should provide this. It makes me wonder why they don't.
Reply With Quote
  #168  
Old 20th October 2017, 10:21 PM
prairielad prairielad is offline
FTDNA Customer
 
Join Date: Feb 2011
Location: Canada
Posts: 1,757
Quote:
Originally Posted by OldFinneyKid View Post

.......

Both files have a number of "null" records where the allele values are "0" rather than A, T, G, or C. Ancestry documents this. There are also some records with values of "I" or "D" -- I can't find documentation of these (ideas?).

The rs identification strings are different for some of the locations; however, the positions on the chromosome are consistent.

........
I and D refer to insertion and deletion

rs numbers merge with others when reference build is updated. one would have to look up rsid to see if has merged with another and thus name change
https://www.ncbi.nlm.nih.gov/snp

attached image on onedrive of merged rsids I have identified
https://1drv.ms/i/s!Al27wnXopRKxhwndtftCo9GljnJU
Attached Images
File Type: png merged report.png (83.1 KB, 2 views)

Last edited by prairielad; 20th October 2017 at 10:34 PM.
Reply With Quote
Reply

Bookmarks


Currently Active Users Viewing This Thread: 4 (0 members and 4 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
AncestryDNA Upload Error... ibuprophen Features Requests & Bug Reports Area 1 25th August 2014 10:59 PM
Y-DNA from AncestryDNA? outland Paternal Lineage (Y-DNA STR) Advanced 2 29th June 2013 05:44 AM
23andMe or AncestryDNA next? Pied_Noir DNA and Genealogy for Beginners 21 25th January 2013 12:28 PM
AncestryDNA Swennilsson DNA and Genealogy for Beginners 268 31st August 2012 08:48 PM
PF vs AncestryDNA briancowings Family Finder Advanced Topics 11 27th June 2012 12:10 AM


All times are GMT -5. The time now is 01:29 AM.


Family Tree DNA - World Headquarters

1445 North Loop West, Suite 820
Houston, Texas 77008, USA

Phone: (713) 868-1438 | Fax: (832) 201-7147
Copyright 2001-2010 Genealogy by Genetics, Ltd.
Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.