Go Back   Family Tree DNA Forums > Family Tree DNA Communications > Features Requests & Bug Reports Area

Features Requests & Bug Reports Area This forum is for customers to talk about features they would like to see. It may also be used to report potential website issues and account bugs, though you may get faster service by sending those through the Open a Request feature found in the Customer Service link at the top of the forum page.

Reply
 
Thread Tools Display Modes
  #151  
Old 11th October 2017, 09:16 AM
chrisbonisa chrisbonisa is offline
FTDNA Customer
 
Join Date: Oct 2017
Posts: 24
Blog Entries: 1
Quote:
Originally Posted by chrisbonisa View Post
You are describing a different change that happened last year. I've got 5 total dna samples from ancestry and 3 different formats:

11/24/2014 v1.0 - 701495 lines
03/01/2017 v2.0 - 668961 lines
08/25/2017 v2.0 - 668961 lines
09/29/2017 v2.0 - 668961 lines
10/08/2017 v2.0 - 650429 lines

These are possibly +1 lines if you count the last line-feed. Clearly the last "v2.0" one is in a different format of some kind and its the only one not accepted by FTDNA. Even "changing" the new format one to the other 2.0 formats by removing new snps and filling the missing snps with no-calls doesn't seem to fix it as FTDNA wont accept that new file either with the same error message, although its possible my method of creating this new file is introducing other changes to the file I'm unaware of. I'm willing to provide the last 2 samples to anyone who has the programming skills to attempt what I did. Let me know...
Just to elaborate for anyone else looking at the technical part of this problem, here are the file sizes unzipped:

11/24/2014 v1.0 - 701495 lines - 18,841,144 bytes
03/01/2017 v2.0 - 668961 lines - 18,110,076 bytes
08/25/2017 v2.0 - 668961 lines - 18,110,076 bytes
09/29/2017 v2.0 - 668961 lines - 18,110,076 bytes
10/08/2017 v2.0 - 650429 lines - 17,595,809 bytes
Reply With Quote
  #152  
Old 11th October 2017, 12:22 PM
genealice genealice is offline
FTDNA Customer
 
Join Date: Oct 2017
Posts: 6
Quote:
Originally Posted by aprilmcg123 View Post
While Dirk's explanation may sound logical when you're only looking at a single file which will not upload, I manage several profiles for my family members and have found that results files from tests run within a single month of each other are of different sizes and some are accepted for upload to ftdna while others aren't. So this can't possibly be due to them using less SNP's for all customers since May 2016.
Hi aprilmcg123,

Oh I agree with you - Dirk's timings are out and it has also occurred to me that Ancestry file sizes are not consistent - I've already put that on this thread or another thread outside this board to which I was posing this question. I thought it was useful that Dirk had confirmed about the SNPs and I'm watching the tekkies here with interest if they can work out a work round for this. I'm not a tekkie - I'm just trying to apply some common sense and get information from others and hopefully something constructive ie a solution to this will eventually be forthcoming.
Reply With Quote
  #153  
Old 11th October 2017, 12:36 PM
CFegley CFegley is offline
FTDNA Customer
 
Join Date: Aug 2017
Posts: 18
Quote:
Originally Posted by aprilmcg123 View Post
While Dirk's explanation may sound logical when you're only looking at a single file which will not upload, I manage several profiles for my family members and have found that results files from tests run within a single month of each other are of different sizes and some are accepted for upload to ftdna while others aren't. So this can't possibly be due to them using less SNP's for all customers since May 2016.
I've previously spoken with Dirk, too, and while he comes across as professional and sincere, the reality is that like others with whom I've spoken, he is simply regurgitating a canned response. If pushed with a logical reason as to why what he is saying is not valid, he will cave, as have many others, including a Francis, a mid-level manager (presumably) who admits that the people in the call center have no knowledge of how the science of DNA works, they are not there to address such questions...their purpose is to explain how you order a test, activate a test, start a tree, update your subscription, etc. It is "not unreasonable" that they might try to figure things out without consulting someone in the area that actually handles the processing of the tests. In fact, they don't even have a way to speak with anyone on the "science" side of the house! Or so they say. Regardless of what FTDNA can or cannot do with the data, and by the way, they are not the only ones who will not accept it, Ancestry owes their customers at least the pretense of respect! I can't tell you how many times their CSRs basically told me I didn't know what I was talking about! If you're going to do something because you don't want your data to be used anywhere else, the have the cahonies to say so and don't belittle my intelligence!

I manage several files whose tests have been processed at different times and their results are consistent with the person (was that you April?) who posted different sizes.

5/2015 - v1.0 - lines: 701,498, size/length: 18,841,299
3/2017 - v2.0 - lines: 668,962, size/length: 18,110,076
8/2017 - v2.0 - lines: 650,430, size/length: 17,595,826
8/2017 - v2.0 - lines: 650,430, size/length: 17,595,826

Personally, from a business standpoint, I think it would be in their best interest to be consistent with what they test and the results they provide. I also think it would be good for them to accept data from other companies...they already have one of the largest, supposedly, databases (yet my mom's FTDNA matches are nearly double of theirs) as it can only help them...AND their customer base. Refusing to do so shows fear and when you show fear, whether it is to your followers or your adversaries, your credibility will tumble.
Reply With Quote
  #154  
Old 11th October 2017, 05:21 PM
prairielad prairielad is offline
FTDNA Customer
 
Join Date: Feb 2011
Location: Canada
Posts: 1,836
chrisbonisa Sent you a PM
Reply With Quote
  #155  
Old 11th October 2017, 05:43 PM
chrisbonisa chrisbonisa is offline
FTDNA Customer
 
Join Date: Oct 2017
Posts: 24
Blog Entries: 1
Quote:
Originally Posted by 462DNA View Post
Like others here, I've been unsuccessful in trying to upload raw data for an AncestryDNA kit which completed its processing on August 22, 2017. When I first downloaded the raw data on the same day it finished, I immediately noticed that the header had a V1.0, something which was mentioned earlier in this thread. However, I didn't try to upload it to FamilyTreeDNA until several days later. When I did so, it was not accepted. I tried the remedies in this thread (changing the header) with no success.

I then decided to try to download a new copy and noticed that the header had changed to match the "older" style which some people had success in submitting. Unfortunately this file was also not accepted.

I noticed that the file size (17,595,826 bytes) did not match the expected 18,110,076 bytes which was mentioned in this thread. In fact, when I first downloaded the raw data and looked at number of lines or SNPs, I noticed it did not match up with the number I expected based on the ISOGG comparison chart.

Yesterday, I decided to download a sample AncestryDNA raw data file posted several weeks ago to the Personal Genome Project in order to compare the lines/SNPs. I found that there are 18532 fewer SNPs in my newer raw data! The older files seem to have 668,962 lines, whereas my newer raw data files only have 650,430.

After doing a basic comparison operation on the two different files, I found that there are 1354 SNPs which are apparently unique to the 17,595,826 byte "broken" file and 19886 SNPs unique to the sample (V2.0) Ancestry file from PGP. Could this be a possible change to Ancestry's chip? If so, it's a shame they're removing more SNPs.

I have to wonder whether these SNPs could have been removed deliberately for medical or other reasons. I have uploaded a list of the unique SNPs to each chip, and a list showing the differences in total SNPs for each chromosome.

Also, today another family member's AncestryDNA kit finished processing and the raw data is the same 650,340 lines.
I've just done a similar comparison using php/mysql database and came up with the exact same results as you with my broken file(2.0b):

v2.0a_count - 668942
v2.0b_count - 650410
diff_count - 18532
v2.0a_missing - 1354
v2.0b_missing - 19886

This suggests its not an error on the part of Ancestry but either a different chip or simply different parsed results for some reason. This also means its most likely FTDNA just not able to handle this new scheme, its probably seeing the "2.0" in the file header and looking for some exact snp schema in the file and since it doesn't match it rejects it. Not counting the new unique snps, this is merely a difference of ~3% which should still be plenty of overlap for FTDNA to handle this, if they would simply try. All other sites are able to accept this type of export without a problem and generate matches from it, I've tried gedmatch, myheritage, wegene, dnaland, etc. FTDNA is the only one erring out...

FTDNA - you are losing money and customers, Ancestry is utilizing this new schema for more and more exports, we need a fix ASAP!
Reply With Quote
  #156  
Old 11th October 2017, 05:45 PM
ltd-jean-pull ltd-jean-pull is offline
FTDNA Customer
 
Join Date: Jun 2016
Posts: 392
Quote:
Originally Posted by KATM View Post
Places like GEDmatch (which has a new section created just for these new chips, called "GEDmatch Genesis") and others like DNA.Land and MyHeritage are using imputation in order to make transferred files work with earlier chip version files. That is not necessarily a good thing; see Roberta Estes' post "Imputation Matching Comparison" (second of what will be three posts on the topic) for one look at it.
I've transferred a couple of kits to MyHeritage from FtDNA. Although each only has 100 or so matches there one or two matches were relatives that had tested there and didn't know they'd get a LOT more matches if they transfer here and to Gedmatch and have since done so.

I also encouraged an adoptee looking for his family to transfer to MyHeritage from Ancestry. I didn't think he had transferred because I couldn't see him, but he had. They share a 64cM segment on Ch2, 62cM on Ch3, 20cM on Ch10, 15cM on Ch16 plus some smaller segments yet MyHeritage doesn't think they're a match!

Last edited by ltd-jean-pull; 11th October 2017 at 05:48 PM.
Reply With Quote
  #157  
Old 12th October 2017, 05:53 AM
Parameswara Parameswara is offline
FTDNA Customer
 
Join Date: Feb 2014
Posts: 171
Quote:
Originally Posted by chrisbonisa View Post

03/01/2017 v2.0 - 668961 lines - 18,110,076 bytes
08/25/2017 v2.0 - 668961 lines - 18,110,076 bytes
09/29/2017 v2.0 - 668961 lines - 18,110,076 bytes
10/08/2017 v2.0 - 650429 lines - 17,595,809 bytes
Maybe AncestryDNA get those samples test in more than one labs. Unlike FTDNA, AncestryDNA don't have their own lab. So, maybe the chips those labs used are not 100% identical each other.

http://labsoftnews.typepad.com/lab_s...a-testing.html

Last edited by Parameswara; 12th October 2017 at 06:00 AM.
Reply With Quote
  #158  
Old 12th October 2017, 06:22 AM
Jim Barrett Jim Barrett is offline
FTDNA Customer
 
Join Date: Apr 2003
Location: Timpson, TX
Posts: 2,057
Quote:
Originally Posted by chrisbonisa View Post
FTDNA - you are losing money and customers, Ancestry is utilizing this new schema for more and more exports, we need a fix ASAP!
The people who are really being hurt, and don't even know it, are those who are buying the down graded product that Ancestry is now selling.
Reply With Quote
  #159  
Old 12th October 2017, 10:30 AM
genealice genealice is offline
FTDNA Customer
 
Join Date: Oct 2017
Posts: 6
Quote:
Originally Posted by CFegley View Post
<<<I've previously spoken with Dirk, too, and while he comes across as professional and sincere, the reality is that like others with whom I've spoken, he is simply regurgitating a canned response. If pushed with a logical reason as to why what he is saying is not valid, he will cave, as have many others, including a Francis, a mid-level manager (presumably) who admits that the people in the call center have no knowledge of how the science of DNA works, they are not there to address such questions...their purpose is to explain how you order a test>>>

If this is true, it is one of the reasons why FTDNA should show itself a cut above and clearly indicate in its website layout that there may be times that not all Ancestry files will be accepted so that folks do not spend large amounts of time trying to upload a file which will not upload. I am not biased towards either Ancestry or FTDNA but it is FTDNA that offer the service.

<<< activate a test, start a tree, update your subscription, etc. It is "not unreasonable" that they might try to figure things out without consulting someone in the area that actually handles the processing of the tests. In fact, they don't even have a way to speak with anyone on the "science" side of the house! Or so they say. Regardless of what FTDNA can or cannot do with the data, and by the way, they are not the only ones who will not accept it,>>>

Well, I am on this thread because I would like to know if there is a way FTDNA will accept the Ancestry file I have and that is my main concern. I know some firms do not accept raw data they have not produced. Which firms advertise they accept all files from named outside firms and then do not?

<<<Ancestry owes their customers at least the pretense of respect!
I can't tell you how many times their CSRs basically told me I didn't know what I was talking about! If you're going to do something because you don't want your data to be used anywhere else, the have the cahonies to say so and don't belittle my intelligence!>>>

In the end, very frustrating as it may be for you and me, it is not up to Ancestry to make their files compatible so that they can be uploaded on to other firmsí servers.

I manage several files whose tests have been processed at different times and their results are consistent with the person (was that you April?) who posted different sizes.

5/2015 - v1.0 - lines: 701,498, size/length: 18,841,299
3/2017 - v2.0 - lines: 668,962, size/length: 18,110,076
8/2017 - v2.0 - lines: 650,430, size/length: 17,595,826
8/2017 - v2.0 - lines: 650,430, size/length: 17,595,826

Personally, from a business standpoint, I think it would be in their best interest to be consistent with what they test and the results they provide.

Interesting - are you saying that the inconsistency is producing sub-standard results?

I also think it would be good for them to accept data from other companies...they already have one of the largest, supposedly, databases (yet my mom's FTDNA matches are nearly double of theirs)

Again, interesting. Are you talking about a comparison between all matches on FTDNA and Ancestry or the quality of the matches?

<<<as it can only help them...AND their customer base. Refusing to do so shows fear and when you show fear, whether it is to your followers or your adversaries, your credibility will tumble.>>>
For myself, as a customer, yes I would appreciate if it was all opened up but if firms choose not to, itís up to them, whatever their reasons!
Reply With Quote
  #160  
Old 12th October 2017, 03:01 PM
ltd-jean-pull ltd-jean-pull is offline
FTDNA Customer
 
Join Date: Jun 2016
Posts: 392
Quote:
Originally Posted by OldFinneyKid View Post
Just curious, but if your file still does not work, do you have the older file with an uncompressed size of 18,400 KB or do you have the newer file with an uncompressed size of 17,686 KB? Does anyone have a file with either of these sizes that still will not upload after the V2 fix?
This post dated the 27th August was the first to give us a hint about what might be causing this current issue with uploading Ancestry autosomal data to FamilyTreeDNA being due to a change in file size -i.e. Missing data.

Some might think that FamilyTreeDNA ought to make it a priority to sort out this issue so that they can upload a file that it is a different format from the the files FtDNA is set up to accept, even though FtDNA doesn't get informed by Ancestry that they're going to change the files. Ancestry can't inform their customer service reps or customers of the file change so I doubt they'd pick up the phone and tell a competitor.

Anyway, back to August 27th. What a shame that some computer geek who works for FtDNA who was stuck at home with water lapping around the porch didn't somehow manage to log-in from home and start working on the coding. And how unreasonable of them to close down the server for a few days meaning that e-mails couldn't be received during the flooding.

I can understand people's frustration, but some people need to get some perspective. FamilyTreeDNA's priority is their PAYING customers who tested with them, and in the last few weeks they've been working in very trying circumstances.

Hopefully it will be sorted in due course, but it wouldn't surprise me if Ancestry continues to muck around with file size. I have no idea why they do it.
Reply With Quote
Reply

Bookmarks


Currently Active Users Viewing This Thread: 3 (0 members and 3 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
AncestryDNA Upload Error... ibuprophen Features Requests & Bug Reports Area 1 25th August 2014 11:59 PM
Y-DNA from AncestryDNA? outland Paternal Lineage (Y-DNA STR) Advanced 2 29th June 2013 06:44 AM
23andMe or AncestryDNA next? Pied_Noir DNA and Genealogy for Beginners 21 25th January 2013 01:28 PM
AncestryDNA Swennilsson DNA and Genealogy for Beginners 268 31st August 2012 09:48 PM
PF vs AncestryDNA briancowings Family Finder Advanced Topics 11 27th June 2012 01:10 AM


All times are GMT -5. The time now is 03:10 AM.


Family Tree DNA - World Headquarters

1445 North Loop West, Suite 820
Houston, Texas 77008, USA

Phone: (713) 868-1438 | Fax: (832) 201-7147
Copyright 2001-2010 Genealogy by Genetics, Ltd.
Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.