teradata vs snowflake
Image: sdecoret/Adobe Stock

The datasphere, defined by the Datasphere Initiative as the complex system encompassing all types of data and their dynamic interactions with human groups and norms, is expected to grow 300% in the next three years, according to scientists at Aston University. Experts from Aston who are working to find new solutions for data storage assure that the situation has reached global crisis levels. There is too much information, and storage challenges are abundant.

SEE: Cloud data storage policy (TechRepublic Premium)

To combat this looming data storage problem, Aston University has received funding to explore novel data storage solutions. Here, we’ll cover some of the challenges they’re working to mitigate and the solutions they’re creating for growing big data storage needs.

Jump to:

The datasphere: Challenges and opportunities

“Simply building new data centers without improving data storage technologies is not a viable solution,” said Matt Derry, lecturer in chemistry within the University’s College of Engineering and Physical Sciences. “The next three years will be crucial. The global datasphere is predicted to increase to 175 zettabytes, with one zettabyte approximately equal to one billion terabytes.”

SEE: Data governance checklist for your organization (TechRepublic Premium)

Data storage concerns include on-site and digital security, data loss, strained budgets, downtime, limited storage resources, lacking scalability, physical and environmental damages, and energy consumption. Few data storage solutions are currently able to balance the budget, capacity and other functional requirements enterprises have today.

How Aston University experts are transforming data storage

Derry and other experts at Aston University have started a research program to develop a new technology that provides data storage surfaces with channels less than five nanometers in width — around 10,000 times smaller than the width of a human hair. The new format and structure is expected to increase the capacity of data storage devices to cope with the amount of data the world produces every day.

Derry will work closely with Amit Kumar Sarkar, a researcher in materials chemistry who is funded by the Engineering and Physical Sciences Research Council.

“We will be exploiting advanced polymer chemistry as a pathway to increase the amount of data that can be housed on storage media,” Sarkar said. “Increasing the efficiency of existing technologies will significantly reduce the need for costly, environmentally damaging construction of new ‘mega data centers.'”

SEE: Folio Photonics innovates in optical data storage for enterprise data centers (TechRepublic)

While Derry has not disclosed if the new system will be optical or magnetic, experimental devices have already created hybrid systems; polymer chemistry is foundational for both types of devices. Polymer chemistry can increase surfaces by creating new layers, allowing systems to reach higher levels of data density.

The data storage market and trends

The Digital Storage Devices Market Size and Forecast report by Verified Market Research reveals the sector, valued at $3.4 billion in 2021, is projected to reach $9.8 billion by 2030.

Digital storage devices include any media used to store and exchange data. They are a critical component of every computing system. Digital storage devices in this market include hard disc drives, solid state drives, memory cards, floppy discs, optical disc drives like CDs and DVDs, and USB flash drives.

Industrial sectors like finance and banking, healthcare and genetics, and information technology and manufacturing are among the fastest-growing data storage users, especially as these industries push into new frontiers. Vendors in cloud, big data, artificial intelligence, supercomputers, business intelligence analytics, Internet of Things and industrial IoT, and 5G technologies are also driving the sector.

SEE: Ravi Pendekanti: How Western Digital’s HDD innovation is lowering total cost of ownership (TechRepublic)

Leading the market, Western Digital, Transcend Information, Toshiba, Teradata Corporation, Sony, Samsung, Intel, SanDisk and Lenovo are all heavily investing in research and development for new data storage solutions.

A growing need for advanced data governance and security in storage systems

Despite the amount of research going into data storage product development, the Datasphere Initiative says it will take more than novel products to solve and manage the global data crisis. The organization explains that policies, governance, compliance and regulations are essential for the new data era.

Additionally, new data technologies will inevitably have to be data-security resilient. The complex global cyber threat environment, rising data and privacy regulations, and consumers that demand enterprises and organizations to manage data professionally and ethically are all important factors that cannot be ignored by the data storage sector.

Read next: Best data recovery software (TechRepublic)

Subscribe to the Data Insider Newsletter

Learn the latest news and best practices about data science, big data analytics, artificial intelligence, data security, and more. Delivered Mondays and Thursdays

Subscribe to the Data Insider Newsletter

Learn the latest news and best practices about data science, big data analytics, artificial intelligence, data security, and more. Delivered Mondays and Thursdays