Announcement

Collapse
No announcement yet.

BAM file generation shut off at least temporarily

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • BAM file generation shut off at least temporarily

    FTDNA's Contact Form no longer offers the choice of "BAM file request". FTDNA sent administrators an ominous email stating:
    ---
    For the next 6-8 weeks we will limit certain services for the Big Y including creation of .BAM files as the team works to resolve severe storage issues that led to substantial hardware purchases and upgrades.

    We will do our best to expedite the process without impacting the already aggressive timeline needed for the new analysis tools, GAP features and new version of Big Y.

    ---

  • #2
    I'd hardly call that ominous. BAM files are pretty big and during the holiday sale there were a lot of Big Y orders done. That google spreadsheet people were sharing with coupons showed that there were 199 Big Y orders. So 200 orders at say 0.5 GB each, that would be 100 GB without counting codes used that weren't on that sheet and all others before the sale.

    The whole process of requesting and waiting for a BAM seemed too manual. Maybe they are trying to upgrade their system so that it can be more dynamic. Automate it a bit more and have BAM files deleted after a set amount of days to have more space again.

    Perhaps it was an issue they have had for a while and never got around to dealing with it until it just was unmanageable. It could help explain why they need so much time. If the waiting period is accurate that means people will have to wait until March for their BAM files!

    Comment


    • #3
      I find it interesting that they bait you with "new analysis tools, GAP features and new version of Big Y" and then won't answer any questions about it when you email them.

      Comment


      • #4
        the

        Originally posted by The_Contemplator View Post
        I'd hardly call that ominous. BAM files are pretty big and during the holiday sale there were a lot of Big Y orders done. That google spreadsheet people were sharing with coupons showed that there were 199 Big Y orders. So 200 orders at say 0.5 GB each, that would be 100 GB without counting codes used that weren't on that sheet and all others before the sale.

        The whole process of requesting and waiting for a BAM seemed too manual. Maybe they are trying to upgrade their system so that it can be more dynamic. Automate it a bit more and have BAM files deleted after a set amount of days to have more space again.

        Perhaps it was an issue they have had for a while and never got around to dealing with it until it just was unmanageable. It could help explain why they need so much time. If the waiting period is accurate that means people will have to wait until March for their BAM files!

        Comment


        • #5
          Doesn't pass the smell test!

          "For the next 6-8 weeks we will limit certain services for the Big Y including creation of .BAM files as the team works to resolve severe storage issues that led to substantial hardware purchases and upgrades."

          Let's see using 800mb (a more than generous amount) for each BAM file, the 200 orders last month would be 160GB which would fit on my laptop with about 500gb to spare. A 3TB WD External drive can be had for about $120 and it would hold approximately 3750 BAM files.

          A 12 TB WD at Amazon costs $449 which is less than 1 BIGY and would hold approximately 15000 BAM FILES. Yes, I doubt they have sold that many BIG Y's.

          I also know they would need to eventually have a bank of these and configure it properly if every FTDNA user had a BAM file but it begs the question, didn't someone think about this long before shutting off the BAM files OR BAM FILE LINKS for EXISTING OLDER BIGY TESTERS who wish to share their data with admins and other external groups like YFULL.

          Not Happy about this at all as is is severely impacting people's productivity and slowing down discovery and progress.

          Drneutrino
          Retired Software Architect

          Comment


          • #6
            Right. I didn't mean to imply that hooking up a few extra hard drives isn't easily doable. Though that does make one wonder how many Big Y tests are in the database. I just used that number as an example of one statistic we barely know. Certainly there were more Big Y tests ordered using codes that never made it onto that spreadsheet plus all other ones before the sale.

            I don't know their setup. They might have storage issues due to all other data and not just the BAM files that will be coming in the future. Perhaps they haven't been keeping up with their digital storage needs the way they should have. Scratch that perhaps, they certainly haven't been keeping up with their storage needs. They may have known this was coming and decided to wait until after the sale to take action so that it would not hurt the number of sales they make.

            At any rate, I didn't see any bad intent on this just sloppiness on FTDNA's part. Either the tech team or management, maybe both. Hopefully they get done sooner rather than later.

            Comment


            • #7
              The only thing I can question is why turn off the ability to order the BAM file, you could keep a record of all the orders coming in for the BAM file and then process them in date order, now if and when they open up shop for the BAM files there will be a flood of them coming in, so I could get me BigY result today and order it tomorrow while others who got their results 6 weeks ago might be placed at the end of the queue if they don't place their orders in time, doesn't seem fair.

              Comment


              • #8
                Yeah. Hopefully automating it is part of whatever they are doing. If so, then BAM file requests won't be needed as the system would have one made automatically.

                Comment


                • #9
                  Originally posted by The_Contemplator View Post
                  If so, then BAM file requests won't be needed as the system would have one made automatically.
                  I agree: The only rational reason for removing the ability to make BAM requests is if the new system will generate BAM files automatically.

                  Comment


                  • #10
                    Originally posted by drneutrino View Post
                    A 12 TB WD at Amazon costs $449 which is less than 1 BIGY and would hold approximately 15000 BAM FILES.
                    I agree that the "storage" excuse doesn't really make any sense. There are a hundred ways to solve a storage problem without removing the primary value (IMHO) of the Big Y for 2 months.

                    The best genuine explanation I've seen is that FTDNA needs to regenerate all BAM files, which will then be compatible with their new system and not their old one. For example, a switch from b37 to b38?

                    Comment


                    • #11
                      .BAM Files Further Delayed

                      Originally posted by lgmayka View Post
                      FTDNA's Contact Form no longer offers the choice of "BAM file request". FTDNA sent administrators an ominous email stating:
                      ---
                      For the next 6-8 weeks we will limit certain services for the Big Y including creation of .BAM files as the team works to resolve severe storage issues that led to substantial hardware purchases and upgrades.
                      ---
                      By my count, 6 weeks elapsed on March 3, and 8 weeks will elapse on March 17.

                      I received my Big Y results on March 9 and immediately sent an inquiry about receiving a link to my .BAM file. The reply:
                      "For the next 2-3 weeks we will limit certain services for the Big Y including creation of .BAM files as we resolve some severe storage issues with ​recent ​hardware purchases and ongoing upgrades while continuing to work on an already aggressive timeline for some user tools for analysis of members' data and project administrator features for the newest version of Big Y.
                      ​T​his became necessary when we filled up a main hard-drive more quickly th​an anticipated from the recent holiday sale.
                      We hope to have it all resolved to everyones (sic) satisfaction by late March."

                      So now, availability of .BAM files is further delayed until "late March".

                      Comment


                      • #12
                        Very frustrating

                        I was recently denied service by another company, after asking for updated batch information for a +$700 Y analysis, and "banned" from every being a customer, by a manager who is apparently 10 years old.
                        My only recourse in achieving more SNP analysis is Big Y, but with this news, not being able to download a BAM for analysis with other tools makes any current price excessive.
                        Sorry to be whiney, but it seems as if every possible roadblock to doing serious Y research is occurring 1st qtr 2016, and I do consider this ominous.

                        Comment


                        • #13
                          Originally posted by clintonslayton76 View Post
                          My only recourse in achieving more SNP analysis is Big Y, but with this news, not being able to download a BAM for analysis with other tools makes any current price excessive.
                          I agree with you that without 3rd-party analysis, the Big Y is simply not worth its price. However, we have been assured multiple times that the removal of BAM file access is only temporary.

                          Comment


                          • #14
                            I too have to agree, someone should tell FTDNA that it is good business practice to keep their customers informed as to their progress of their systems up-grade, the 6 -8 weeks period they claimed that it would take has come and gone and still no information as to what’s happening, even if it’s a delay it would be nice to know rather than being keep in the dark about everything.

                            Comment


                            • #15
                              If you can believe, it looks like it's going to be another 2 weeks before we can start requesting Bam files again, not happy because of their track record you just know that it will blow out again.

                              Comment

                              Working...
                              X