Storage Evolution Powering Computer Innovation—unfolding waves of creative destruction

Like many other significant innovations, the computer also started the journey in primitive form. The advancement of major building blocks has been driving the evolution from a roomful of machines to handheld supercomputers like smartphones. Storage is among the major building blocks of computers. Like the computer, storage has also been going through an evolution. Its long journey of advancement is not only driven by the incremental improvement of a single technology core. Instead, it kept changing the technology core, from paper to semiconductor. Along with the advancement of storage and also the computer as a whole, the switching of technology core also fueled the unfolding of creative waves of destructions. It also created discontinuity, offering opportunities for new entrants, and also caused a disruptive effect on incumbent firms. From the perspective of Innovation management, the storage evolution powering computer evolution is full of lessons. 

Underlying drivers for storage evolution powering the computer innovation

The computer has a long history. To get the arithmetic tasks done better, in 2400 BC, Romans developed an abacus. Since then, there have been many major developments reaching the unveiling of electronic computer ENIAC in 1945. ENIAC stands for Electronic Numerical Integrator and Computer. In today’s standard, it was an extremely primitive machine– weighing 30 tons, using 200 kilowatts of electric power, and containing over 18,000 vacuum tubes. Despite having the potential of a bright future, this primitive machine failed to create a willingness to pay among many customers. Similar to many other great innovations like the Internet, airplane, and cellular communication, ENIAC became eligible to work for the US military. Performing ballistic trajectory calculations for the United States Army was one of its main tasks. 

To counter increasing war challenges, the military was asking for the advancement of this machine. To meet growing requirements, contractors kept focusing on major building blocks, including storage. However, this role was taken over by competing innovative firms at a later stage. The focus of storage performance improvement was in addressing volatility, mutability, accessibility, addressability, capacity, energy use, security, space requirement, and overall performance.    

The emergence of competition force in storage evolution for powering computer innovation

In the 1940s and 1950s, computer making and advancement mission was dominated by inhouse development teams and contractors of the US department of defense and other handful large customers. In those days, customers’ requirements to increase the performance to address their mission challenges drove the progression of the computer as a whole and storage in particular. However, in the 1960s, the world started observing the emergence of innovating firms. As opposed to waiting for customers to tell them what the advancement to be made, they started risking their time and money to innovate machines for alluring customers. Hence, door to benefit from innovation started to get wider.

The entry of an increasing number of innovative firms led to the competitive race of performance improvement. For both expanding the scope of use of computers and also to sustain the innovation in the competitive market, the progression of computer storage accelerated. Consequentially, the force of competition to profit from ideas became dominant to drive storage evolution for powering computer innovation. The competition ended up in the race of offering better quality storage at decreasing prices, which led to the generation and exploitation of ideas for storing an increasing number of bits in the same amount of material. 

Moreover, the competition force started to intensify due to innovators’ entry as start-ups.  However, most of them emerged to commercialize ideas developed in university-based defense-funded R&D projects in US universities. On top of it, Japanese and American large firms like IBM, Toshiba, Hitachi, and Fujitsu kept refining ideas for better quality storage. In fact, many of those ideas originated from defense-funded R&D projects in US universities. 

Storage evolution began with mechanical technology–Punch cards and Punch tape 

We know computers as advanced electronic machines. However, the journey of storage for electronic computers started with the support of mechanical technology. In fact, IBM adapted punch cards and paper tape technology from the textile industry. After more than 100 years of the invention of the Jacquard loom, IBM adapted and patented punch card input devices from it for modern computers. Besides, IBM made a fortune from punch cards over almost 30 years. As late as the mid-1950s, punched card sales made up 20 percent of IBM’s revenues and an astonishing 30 percent of its bottom line. Unfortunate, during this time, there was very little improvement in computer storage as IBM had a virtual monopoly.   

In fact, perforated paper tapes constructed from punched cards were first used by Basile Bouchon in 1725 to control looms. In the 20th century, computer makers adapted paper tapes for storing data. Even in the 1970s and 1980s, paper tape found common use to transfer binary data for incorporation in either mask-programmable read-only memory (ROM) chips or their erasable counterparts – EPROMs. 

The long journey of evolution of storage through magnetic technology 

To increase the speed and reduce manual interventions, the military was looking for greater automation in the memory sub-system of computers. This urge led to the beginning of the exploration of materials’ magnetic properties to store and retrieve information. Hence, it started a long journey of innovation.

Magnetic drum memory

To fulfill the requirement of a US NAVY contract, Engineering Research Associates (ERA) delivered a stored program computer-ATLAS in 1950. It used magnetic drum memory. It stored information on the outside of a rotating cylinder coated with ferromagnetic material and circled by read/write heads in fixed positions. However, in Austria, Gustav Tauschek 1932 invented drum memory based on magnetic data storage principles. Tauschek’s drum memory was capable of storing 62.5 kilobytes of data. It kept finding its use still in the 1980s. The purchase of patents of drum memory from Tauschek let IBM commercialize drum memory as part of its first mass-produced computer, the IBM 650. 

Magnetic core

The access time of data in the magnetic drum used to depend on the position. To overcome it, random-access magnetic core memory was developed. It remained in use as a dominant form of data storage for 20 years between about 1955 and 1975. Core memory uses toroids of hard magnetic material as transformer cores. Each core stores one bit of information. A core can be magnetized in either the clockwise or counter-clockwise direction. Cost of material and high labor requirement to manufacture made core memory expensive.

By the late 1960s, a density of about 32 kilobits per cubic foot was typical. However, continued Incremental innovation led the price to decline from $1 per bit to about 1 cent per bit. Moreover, the journey of magnetic core memory witnessed bitter intellectual property lawsuits, resulting in IBM buying the patent outright from Wang for US$500,000, and IBM paying MIT $13 million for rights to Forrester’s patent.

Hard disk 

In 1953, the engineers at IBM’s San Jose California laboratory invented the hard disk drive, made from magnetic disks. Subsequently, in 1957, it’s found its commercial use in the shipment of production of IBM 305 RAMAC system, including IBM Model 350 disk storage. In fact, hard disk data storage technology was highly amenable to growth, leading to continued quality improvement and cost reduction. The scope of expanding the market and the opportunity of profiting from ideas encouraged the entry of a growing number of firms. Consequentially, it grew as a large industry. At its peak in 1985, the hard disk industry had 75 active manufacturers. The volume also kept growing, reaching 22 million units and $23 billion in revenue in 1989.

The remarkable journey of HDD experienced its first-ever decline in units and revenue in 2001, due to the uprising of flash memory. Subsequently, the number of industry participants decreased to 6 in 2009 and to 3 in 2013. In its long journey, HDD witnessed increasing data density and falling cost per bit. Even in the recent past, it kept experiencing this trend. For example, from January 2015 to January 2017, the average cost decreased 26% from $0.038 to $0.028 – just $0.01 per gigabyte.

Magnetic tape, Floppy diskette, and CD ROM fueled Storage Evolution for Powering Computer Innovation 

For recording and playback of audio and music, magnetic tape was developed in Germany in 1928. In the computer, it was first used to record computer data in 1951 on the Eckert-Mauchly UNIVAC I. Subsequently, it became very popular for bulk backup storage. In the 1970s, audiotape became popular among personal computer makers.

With the support of R&D work in the late 1960s at the San Jose facility, IBM introduced floppy diskette into the market in an 8-inch format in 1972. In the 1980s, it became trendy in the PC industry. Its 40 years-long popularity was taken over by the uprising of NAND flash technology in the form of USB flash drives and memory cards. In the 1990s, the computer industry witnessed the emergence of optical property-based data storage technology CD ROM. In fact, it caused a high-level substitution effect on the floppy diskette. Subsequently, its popularity in the computer industry was taken over by flash memory.  

Emergence and growth of Flash memory

The long journey of data storage in the computer industry witnessed the arrival of over half a dozen major innovations around three technology cores—mechanical, magnetic, and optical. Despite having a substitution effect, however, many of them coexisted for a long time. Finally, it appears that the journey culminates into surrendering to a single technology core. This is about semiconductor-based flash memory. While working for the Toshiba corporation, Fujio Masuoka invented flash memory in 1984. This storage technology is based on a grid with two transistors forming two gates: the floating gate and the control gate. Control of these gates through the application of voltage leads to data storing and erasing. There has been rapid growth in storage density and cost reduction of flash memory. Upon substituting floppy diskette and CD ROM, it has started to replace HDD in many laptop computers. In fact, it has highly disruptive effects.

The computer data storage industry appears to be heading for an all-flash future. Upon causing Creative Destruction to portable and HDD market segments, the emergence of 30TB to 100TB bulk flash drives in 2018 has already started to take over secondary storage. In fact, within 30 years, the volume of NAND Flash has reached over $50 billion. Such an uprising also faces the possibility of substitution. For example, the progress of data storage in DNA nucleotides might open a better alternative.

For instance, in March 2017, scientists reported 215 petabytes of data storage per gram of DNA. In addition, there have been other developments in progress.  For example, the joint R&D effort of Intel and Micron has resulted in the development of new non-volatile semiconductor memory. It is called 3D XPoint. It’s being claimed that this type is both 1,000 times faster and has 1,000 times greater endurance than NAND flash.

Storage evolution powering computer innovation is full of waves of creative destruction

This long journey of storage evolution has played a key role in evolving computers from a large primitive machine to a high-performing hand-held supercomputer. Of course, in the beginning, defense contracts played a key role to kick-starting the process. Subsequently, profit-making competition kept unfolding waves of creative destruction. These waves kept offering us better alternatives, leading to a flash drive. But this process also witnessed discontinuities caused by the emergence of technology cores, which led to an opening window for new entrants and also disrupting incumbent firms. For example, hard disks created entry opportunities for many. Subsequently, the uprising of flash compelled many of them to leave the industry. Moreover, the uprising of cloud-based storage service delivery is also disrupting the business model of computer storage, particularly in the portable section.  Hence, it demands a strong focus on management focus.