petabyte

It wasn't that long ago that data storage devices were measured in gigabytes and terabyte hard drives seemed exotic. Now that terabytes are commonplace, the next unit of data measurement you're likely to hear about is the petabyte.

In data storage, terms have been defined out to the yottabyte (1024 bytes), or the yobybyte (280 bytes) if you prefer binary to decimal. Having defined those terms, data storage capacities are now working through them one by one. Even PCs have gone from the kilobyte through the megabyte and are now featuring gigabyte RAM and terabyte storage.

Enterprise storage systems are starting to leave the terabyte behind, moving into petabytes and toward the exabyte stage. A petabyte (PB) is 1015 bytes of data, 1,000 terabytes (TB) or 1,000,000 gigabytes (GB).

Petabytes and Pebibytes

A petabyte is sometimes defined as a 250 bytes. This is technically incorrect. There are two different systems of measuring bytes: the Base10 SI system and the Base2 IEC system. While 250 bytes is close to 1015, it is higher and has a different term – pebibyte (PiB). Computing is based on the binary system, rather than decimal. At low levels, the SI and IEC terms are, in practical terms, interchangeable. 103 equals 1000 and 210 equals 1024 so there is only slightly more than 2% difference between the one kilobyte (1000 bytes) and one kibibyte (1024 bytes). And as storage systems are never run to within 2% of their capacity, there is no operational difference between the two.

There is greater divergence, however, as we get into higher orders of magnitude. When you get up to the scale of one petabyte, it does make a difference with a pebibyte (250) being 12.6% larger than a petabyte. Here is a full rundown of the two systems:

 SI decimal prefixes IEC binary prefixes Percentage Difference IEC/SI Name Value Name Value kilobyte (kB) 103 kibibyte (KiB) 210 2.4% megabyte (MB) 106 mebibyte (MiB) 220 4.9% gigabyte (GB) 109 gibibyte (GiB) 230 7.4% terabyte (TB) 1012 tebibyte (TiB) 240 10.0% petabyte (PB) 1015 pebibyte (PiB) 250 12.6% exabyte (EB) 1018 exbibyte (EiB) 260 15.3% zettabyte (ZB) 1021 zebibyte (ZiB) 270 18.1% yottabyte (YB) 1024 yobibyte (YiB) 280 20.9%

When specifying capacity of a large scale system, therefore, it is important to make sure not to conflate the two different systems of counting data storage space. Also, be sure to determine exactly how a particular vendor is counting that space. In enterprise systems, raw disk space alone is not the sole determining factor. How the drives are partitioned, the sector size, the level of RAID (Redundant Array of Independent Disks) or RAIN (Redundant Array of Independent Nodes), any compression or deduplication technologies used and other factors must be considered and compared to the end use case to see how efficiently that storage capacity lines up with the usage scenarios.

Petabyte storage vendors

Petabytes per se are not sold, but there are vendors who sell massive storage systems, including:

Teradata Inc. The Teradata Database 13.10 is a data warehousing database designed to scale from fewer than 10GB up to multiple PB of data.  At the top end, the database can have 186 PB of data and 4096 nodes. The company also has data appliances, private cloud services, MapReduce appliances for big data analytics, and other products and services.

IBM Scale Out Network Attached Storage (SONAS) supports up to 256 file systems. Billions of files and up to 21PB of storage in a single file system.

EMC Isilon Network Attached Storage can store up to 15PB of data and a trillion files in a single namespace.

Hewlett-Packard IBRIX X9000 Storage: Modular architecture that grows up to 16PB in a single namespace.

Hitachi NAS Platform (HNAS) works with Hitachi's SAN storage. Each file system can have up to 256 TB and 16 million objects, with up to 16PB total usable capacity.

Panasas ActiveStor: rackmount appliances containing 20 2TB or 3TB SATA drives. Can scale up to 6.6PB and 4 billion objects per file system.

Petabyte technology

Apache Hadoop is a project developing open-source software that is commonly used for handling Big Data (http://hadoop.apache.org/).

The Petascale Data Storage Institute hosts an annual Parallel Data Storage Workshop (http://www.pdsi-scidac.org/), held in conjunction with the IEEE’s fall Supercomputing workshop.

Next steps

Looking Behind the Big Data Storage Buzz

How Facebook Is Handling All That Really Big Data

IBM Storage Buying Guide

Panasas Storage Buyer's Guide

In-Memory Analytics Buyer's Guide: Oracle Big Data/Exalytics Appliances vs. SAP HANA

Big Data Buyer's Guide, Part Two: IBM, SAS, Pentaho and More

Top Terms
• 1

agile software development

Agile software development is an alternative to traditional project management where emphasis is placed on empowering people to collaborate and...

Read more »

• 2

ERP module - Enterprise Resource Planning module

ERP (Enterprise Resource Planning) software consists of multiple enterprise software modules that are individually purchased.

Read more »

• 3

BYOD - bring your own device

A phrase that has become widely adopted to refer to mobile workers bringing their own mobile devices, such as smartphones, laptops and PDAs, into...

Read more »

Connect with Webopedia

• VMware Virtual SAN Explained

Here's what you need to know about VMware's vSAN software-defined storage (SDS) offering.

Read More »

Did You Know? Archive »

• Webopedia's Cloud Dictionary: 50 Cloud Computing Terms to Know

From planning a private cloud project to finding an online cloud storage provider, Webopedia's A-Z Cloud Computing Glossary will help you understa..

Read More »

• 10 Essential Social Networking Tools to Save Time

These free social networking tools that can help you manage numerous social networking and bookmarking accounts, plus save you time by aggregating yo..

Read More »

• 5 Open Source Tools for Web Developers

There's always something new in open source software. One trend is the sheer volume of projects surrounding JavaScript and Web development.

..

Read More »

• Server Types

Different servers do different jobs, from serving email and video to protecting internal networks and hosting Web sites. Learn about the many type..

Read More »

• An Introduction to Data File Formats and Their File Extensions

Ever encounter a file extension you've never seen before? Check this page for more 3,700 data formats and their corresponding file extensions.

..

Read More »

Quick Reference Archive »