Affy to Illumina conversions complete?

[similligan]

I manage one of those missing 114 conversions and I'm OK with it. They needed a new DNA sample and the participant, for good reason, didn't put it high on his list of priorities. I'll just be getting a later start than the rest of you.

Susan

[Javelin]

Right, I suspect those 114 are all still to go. It shouldn't make too much of a difference for most people's match lists, unless you are one of the unfortunate 114 who have been waiting three months and, as of today, will have zero Affymetrix and zero Illumina matches.

[mkdexter]

There are at least some kits from batch 400 still to be converted, possibly all 100-some, but I'm not sure.

114 was the actual number as of last Friday.

[mkdexter]

I know FTDNA was working on some kind of an announcement. I am guessing it will be today that we see it.. just guessing its today though.

MD.

[Javelin]

Or are there still some conversion results to be posted? Are they all finished? Where is the large group of results that Matt referred to early last week? Have those been posted with no comment be any forum members?

There are at least some kits from batch 400 still to be converted, possibly all 100-some, but I'm not sure.

[CAM]

So I guess with Matt's detailed explanation, and no further Illumina conversion results in several days, we are all satisfied??

Or are there still some conversion results to be posted? Are they all finished? Where is the large group of results that Matt referred to early last week? Have those been posted with no comment be any forum members?

Will we ever know for certain when all of the conversions have been completed, posted and parsed? I am not expecting much aside from closure, a point at which I know that the only new Illumina results will be the result of newly purchased kits.

John, No not satisfied but not much we can do. One of the matches I lost was a 4th Cousin - Distant Cousin - 26.38 10.33 I guess she fell below the number to keep her as a match and 52 matches gone. My uncle got a few and my father got none I lost.

I have not had any matches that have not been conversions for I don't know how long. New matches what are those not with my kit they are far and not in between.

I am more mad about when they told us about the switch to Illumina since I had put my uncle in for FF and it was not done for months later and with Illumina not Affy. I don't know if he would have gotten matches under Affy at that time that I had and then when they disapered I would have at least known what side of the family they were on.

Cheryl

[JPHutchins]

Are all of the conversions complete?

So I guess with Matt's detailed explanation, and no further Illumina conversion results in several days, we are all satisfied??

Or are there still some conversion results to be posted? Are they all finished? Where is the large group of results that Matt referred to early last week? Have those been posted with no comment be any forum members?

Will we ever know for certain when all of the conversions have been completed, posted and parsed? I am not expecting much aside from closure, a point at which I know that the only new Illumina results wil be the result of newly purchased kits.

[JohnG]

Wow

Thank you Matt, for going into this in such a detailed and specific way. This looks like the start of a research paper! It is what I really hoped FTDNA would have done as part of the process of conversion.

I want to think about this awhile!

I do have a basic question. Which matches are useful for genealogy?

John Olson published a list of known relationships on this forum. It had about 300 matches of known relationship from 1st cousin to 5th. I expect most of the first and second cousins were known before testing, and they account for half the list. Of the remaining 150, about 40 are 5th cousins. I wonder how many of those matches persist under Illumina?

With 9 kits of family members, we have made two real breakthroughs. One of those was 6th cousins, who were present in Affy and not in Illumina. The other was a 5th cousin once removed (two ways) who did match my brother and a 1st cousin, and still does. I suspect it is that marriage of 1st cousins a few generations back that did that.

My wife helped a 5th cousin make a breakthrough because she has a common paper ancestor with a person he matches but she does not.

We had an 8th cousin match, too, with a common ancestor in 1680.

I guess our projects are mostly out there at 4th and 5th cousins and the less strong matches seemed to help that kind of research. Especially when they match in common. It is indicating where to look, and sometimes who to look at.

I know others are looking for 2nd or 3rd cousins, and maybe I have more of those to find than I think. My larger projects are probably going to be solved with 4th and 5th cousins.

So, I understand what you are reporting and how it explains the change in matches. I am less sure if that change is positive or negative, and for whom.

[mkdexter]

Continued....

Matches A, B and C Affy compared to me in the chromosome and positions FF said we matched:

Chr 17 has a Single Strand Match of length 761 from position 34276 to position 5633723 ( 5.60 Mb) (7 non-matching SNPs treated as matching)

Chr 5 has a Single Strand Match of length 629 from position 174950255 to position 180268277 ( 5.32 Mb) (6 non-matching SNPs treated as matching)

Chr 21 has a Single Strand Match of length 826 from position 41219289 to position 46915057 ( 5.70 Mb) (7 non-matching SNPs treated as matching)

The above illustrates the matches are borderline to start with, all having a large number of non-matching SNPs for small areas of shared DNA.


Comparing Affy to Illumina from the second set, Match B (Affy matched and Illumina did not):

Match B PIKE AFFY COMPARE
Chr 5 has a Single Strand Match of length 629 from position 174950255 to position 180268277 ( 5.32 Mb) (6 non-matching SNPs treated as matching)

Match B PIKE ILLUMINA COMPARE
Chr 5 has a Single Strand Match of length 685 from position 177712172 to position 179857818 ( 2.15 Mb) (5 non-matching SNPs treated as matching)

Note Illumina does test past position 180268277 but this match didn't pick it up.


I'm adding this here to illustrate that all SNP tests will have in them some number of differences and that is why the matching process has to have within it some tolerances as to how many non-matching SNPs are allowed.

worse case my FF-Iv1 to my RF-V3
Chr 1 has a Double Strand Match of length 57997 from position 1404962 to position 247185615 (245.78 Mb) (15 non-matching SNPs treated as matching)

different reported SNPs my FF-Iv1 to my RF-V3,
693890 SNPs appear in both files

0.528 % (3661) of these 693890 common SNPs are NoCalls (i.e. NoCall in at least one file)

0.018 % (123) of the 693890 SNP results differ

89 differing SNPs are both homozygous
19 differing SNPs are homozygous in File1 but not in File2
12 differing SNPs are homozygous in File2 but not in File1

RF V3 and FF Illumina V1 are tested using the same chip and process, just different times and companies. This is why some non-matching SNPs are allowed by the algorithms, otherwise these errors would prevent real matches from showing up.

My opinion and findings are that the combined borderline, many non-matching SNP Affy matches just didn't pass the higer resolution, differing SNP Illumina test. I think Illumina has some borderline matches too and if re-tested with a different platform they would also disappear.

MD.

[mkdexter]

I've completed checking data.

I've completed my own check on a few of my missing Affy matches.

I obtained three sample of FF Affy raw data from those I matched in FF Affy but not in FF Illumina.

Using David Pike's utility I found that in the stated segment where FF said I matched them, the longest block in other words since it define the match and prediction, there were 7 non-matching SNPs.

The Affy matches were allowed to meet criteria with so many non-matching SNPs because Affy allows a large tolerance for this. David Pike's utility calls it "Treat as matches any non-matching SNP that is at least xx SNPs away from its nearest non-matching SNP". In order to see it I had to lower the 150 SNP limit to 20 in the Pike utility.

This established the actual readings used by FF to make the original Affy matches. I then noted the chromosome, positions and SNP readings and compared them to the chromosome map table in Family Finder.

Next I compared thise same areas in the Illumina raw data using the same tolerance for non-matching SNPs.

I found that Illumina also saw the same number of non-matching SNPs but in addition and due to the difference in the SNPs actually tested, the original segments were usually broken or split so that in the Illumina test, the criteria for a match was not met.

What does this tell me then? First, the Affy matches I reviewed all had in them non-matching SNPs in the comparison. They were borderline in other words. Second, the differences in the tested SNPs and the resolution of the Illumna test created a situation where, if the test came in borderline, it came out as a non-match.

These matches of mine are indeed somehow related although I don't know exactly how yet, because their relatives (cousins) are still matches to me in Illumina, as they were before. The problem is my missing Affy matches happened to be right on the edge of even matching me to begin with. The Illumina test being a different test did not consider them a match.

In the past month I've also ran tests proving that once the segments are small enough they begin to persist over many generations as the same size and sum. I observed this in my granddaughter's FF test where my mother and my granddaughter, a 4 generation difference, have some of the same matches who are listed by the same range and prediction; either 4th to distant or 5th to distant. I bring this up because it illustrates a possible factor in the Affy tests, that being that my 5th to distant matches that I've lost in Illumina could very well be much farther distant than first thought. It is very possible and probable the Affy matches I've lost have a common ancestor to me in the 8th generation and beyond range.

I also know for a fact that my mother lost a known Affy match who is 9 generations from her while I did not lose this same match. This most likely occurred because as with any test, whether Affy or Illumina there will always be a small number of miscalls and/or microdeletions that affect the outcome. I've compared my FF Illumina to my RF V3 data and even there I see some differences in the results yet these two tests are both ran on a Illumina Omni-Express chip.

I'm finished running my comparisons. I believe as we all have stated already that the Affy test allowed some borderline matches through, who were actually real matches but were very distant, while Illumina with its differences such as different SNPs tested and more SNPs tested, did not. Although there might be a "false positive" or two out there, I think most of them are simply very distant and borderline matches to begin with that Illumina said "no" to.


MD.

[nolnacsj]

Affy/Illumina cousin results for two sisters

I have a Affy match who has yet to show up in my Illumina, but her sister showed up as a match with me on Illumina.

Judy

[gtc]

I've mentioned that a goodly number of Affy matches that don't show up in Illumina could very well be false positives.

I'm inclined to accept that as the most likely reason that I have "lost" 42% of my Affy matches.

[duse]

The genes come together and split and come together later on. The pseudo-segments have to be the same in several people to make a match. That means the accidental reshuffling has happened in at least two cases. I think I may have cases where it showed up in a multiple cases - some of those 8 cM segments that a dozen matches shared. So they are pseudo in that they do not come from one individual to their child, but maybe they come from a shared ancestral population down to many descendents?

This is why I have become extremely frustrated. I had my half sister tested and I am still struggling to find good matches on my fathers side. Just using FTDNA, I see many matches until I use Gedmatch and I see my half sister's matches or my matches in her Gedmatch. I love Gedmatch, but I must not be using it like I should. Thanks for any help.

[Ann Turner]

One of my major learnings from Family Finder is how interrelated all of us are from even a vague huge grouping like Americans whose families have been here 250 years. I find a 5th cousin and they are also my wife's 4th cousin. My wife is my 1st cousins 5th cousin, etc. And I am talking of paper trails as well as dna matches.

The genes come together and split and come together later on. The pseudo-segments have to be the same in several people to make a match. That means the accidental reshuffling has happened in at least two cases. I think I may have cases where it showed up in a multiple cases - some of those 8 cM segments that a dozen matches shared. So they are pseudo in that they do not come from one individual to their child, but maybe they come from a shared ancestral population down to many descendents?

What you say is conceptually possible. We need a better vocabulary, but perhaps we could call those overlapping segments in distinction to pseudo-segments, where I am talking about tiny bits and pieces. But we really need phased data to demonstrate what's happening.

[Ann Turner]

quirky case

I've mentioned that a goodly number of Affy matches that don't show up in Illumina could very well be false positives.

On the flip side, I have an example I would consider a true positive, but it didn't pass the Illumina test due to a "perfect storm" of circumstances. Much of this analysis was expedited by David Pike's utilities at

http://www.math.mun.ca/~dapike/FF23utils/het.php

It was a robust 12 cM, 8.75 Mb match with about 2500 SNPs, originally found at 23andMe and confirmed in the Affy test at FTDNA. I had also checked it with some haplotype data, not just genotypes.

The 23andMe test had no discrepancies in the region, while the FTDNA had five. This illustrates why FTDNA allowed a certain number of mismatches to compensate for the error rate on the Affy chip (still low in absolute terms).

The FTDNA Illumina chip added more SNPs, and the original segment split in two parts, 6.11 Mb (no mismatches at all) and 2.75 Mb. These two pieces were separated by just 9500 bases and had three mismatching SNPs in a row. That can be a sign of a microdeletion, where one of the parties actually has just one allele, but it is reported as a homozygous genotype.

The original test had just one SNP in the "microdeletion" region, and it was a match. However, the allele is very common, found in 95% of Europeans, so it's not surprising that it passed muster.

The 6.11 Mb section in the new Illumina test is 7.65 cM in length, so presumably it just missed the FTDNA threshold, which seems to be 7.7 cM. (The cM value is according to the Rutger's map, which may or may not be what FTDNA is using but is probably close).

So bottom line: the additional Illumina SNPs revealed a microdeletion, which chopped the segment into two parts, and the larger part just missed a threshold.

I'm posting this, not to get your hopes up about all of your missing Affy matches, but to illustrate that it sometimes takes digging down deep in the raw data to understand what's going on. There's no way to develop an algorithm to handle quirky cases like this.