AWS DataSync customers

The partnership with AWS continues to help our customers access lossless, high-quality historical data. With over 30PB of data now migrated, thanks to both teams at AWS and LSEG, leveraging sophisticated tools and processes, we can continue to add new global multi-assets venues as customer demand continues to grow for LSEG PCAP data.

With AWS DataSync, TMAP Mobility Co., Ltd. was able to quickly and reliably transfer PB scale Cloudera HDFS data within 2 months. AWS DataSync’s native support for HDFS helped us meet our tight project launching schedule.

We at FORMULA 1 used both AWS DataSync and AWS Storage Gateway services to provide disaster recovery for various data streams. We used AWS DataSync to implement a DR solution for our raw video archive. We chose AWS DataSync as it allows us to sync our current data, totaling 400 TB, to Amazon S3 in a way we could control and it enables us to keep any new data in sync. With AWS DataSync we were able to provide DR by syncing data from our on-premises file server to Amazon S3 at an average of 4 TB per day, and did so with encryption during transfer and at rest. The settings and controls provided by AWS DataSync allow us to control when and how much data we want to sync without harming any other workloads.

DataSync gave us the ability to streamline our data migration project saving a significant amount of time, not to mention lots of headaches! We were able to complete our migration of 2.5 billion files to Amazon S3 that helped the PGA TOUR reduce overall costs.

Lower cost and scalability of DataSync vs other tools are the outliers as to why we chose it for migrating petabytes worth of HDFS Data. Ease of setting it up via Infrastructure as Code made it easier to deploy and maintain in AWS Cloud.

AWS DataSync proved to be the powerful migration service that ClearScale needed in order to move huge amounts of frequently changing data. The AWS, fully-managed service made the execution of our client’s migration plan much easier and more efficient than if we would have used custom scripts or a competing migration service.

The combination of fast and secure data transfer by AWS DataSync and the large storage capacity of Amazon S3 allowed us to smoothly deliver genome data, which is becoming increasingly large. We will continue to build an analysis platform using AWS for personalized medicine using whole genome sequencing, which is expected to increase in the future.

We have genome sequencers running daily and need to transfer files a minimum of two times each day from on premises to Amazon S3 in an accelerated and efficient manner, in order for us to analyze and process the sequencer data in AWS. AWS DataSync serves this purpose perfectly as we are able to easily schedule and automate the file transfer process, avoiding the need for costly commercial transfer tools. In addition to moving the sequencing data to the cloud for analysis, we also use DataSync for a number of other projects and needs: to migrate active data to AWS, to transfer data to the cloud in order to securely share data with external collaborators, and to create an additional copy of our invaluable data in the cloud for business continuity. DataSync provides us with highly performant, reliable, and secure data transfer capabilities that we require in order to help solve the world’s biggest health challenges.

Our petabyte scale data migration journey from on-premises to AWS was accomplished swiftly with minimal effort and was completely self-managed with AWS DataSync. This solution is a game changer!

A big challenge that we face is the integration of the wet labs with the computational aspect of our research. We use Storage Gateway and DataSync to synchronize our on-premise wet labs with our Amazon storage. By having our scientists immediately be able to save their files directly to the cloud, they can go on with the next experiment without having to wait for the transfer times and they never run out of space.

Our architecture and the critical nature of our business required us to test our business capabilities on AWS over a series of trials before we could flip traffic to AWS. We, therefore, set a design goal of being able to shift traffic from on premises to AWS or back within minutes. Our hybrid file and block-based replication approach allowed us to accomplish just that, and DataSync played a critical role in our successful migration.

Prior to learning about AWS DataSync, I had spent over a month creating, testing, and iterating on new scripts to get our backup files into Amazon S3. These scripts needed to be deployed on all of our servers, and were difficult to centrally manage. By transitioning to DataSync, I was able to simplify and automate my backup management. Apart from its simplicity, DataSync also provided additional functionality, such as monitoring and error checking, that was quite valuable. It took less than one hour of my time to set up and get going, and we use DataSync every night to transfer 5 – 50 TB of data, which is roughly 3,000 files. For the task that I needed it for, DataSync was absolutely spot on.