The October 2022 issue of IEEE Spectrum is here!
IEEE websites place cookies on your device to give you the best user experience. By using our websites, you agree to the placement of these cookies. To learn more, read our Privacy Policy.
It should come as no surprise that recent advances in big-data analytics and artificial intelligence have created strong incentives for enterprises to amass information about every measurable aspect of their businesses. And financial regulations now require organizations to keep records for much longer periods than they had to in the past. So companies and institutions of all stripes are holding onto more and more.
Studies show [PDF] that the amount of data being recorded is increasing at 30 to 40 percent per year. At the same time, the capacity of modern hard drives, which are used to store most of this, is increasing at less than half that rate. Fortunately, much of this information doesn't need to be accessed instantly. And for such things, magnetic tape is the perfect solution.
Seriously? Tape? The very idea may evoke images of reels rotating fitfully next to a bulky mainframe in an old movie like Desk Set or Dr. Strangelove. So, a quick reality check: Tape has never gone away!
Indeed, much of the world's data is still kept on tape, including data for basic science, such as particle physics and radio astronomy, human heritage and national archives, major motion pictures, banking, insurance, oil exploration, and more. There is even a cadre of people (including me, trained in materials science, engineering, or physics) whose job it is to keep improving tape storage.
Tape has been around for a long while, yes, but the technology hasn't been frozen in time. Quite the contrary. Like the hard disk and the transistor, magnetic tape has advanced enormously over the decades.
The first commercial digital-tape storage system, IBM's Model 726, could store about 1.1 megabytes on one reel of tape. Today, a modern tape cartridge can hold 15 terabytes. And a single robotic tape library can contain up to 278 petabytes of data. Storing that much data on compact discs would require more than 397 million of them, which if stacked would form a tower more than 476 kilometers high.
It's true that tape doesn't offer the fast access speeds of hard disks or semiconductor memories. Still, the medium's advantages are many. To begin with, tape storage is more energy efficient: Once all the data has been recorded, a tape cartridge simply sits quietly in a slot in a robotic library and doesn't consume any power at all. Tape is also exceedingly reliable, with error rates that are four to five orders of magnitude lower than those of hard drives. And tape is very secure, with built-in, on-the-fly encryption and additional security provided by the nature of the medium itself. After all, if a cartridge isn't mounted in a drive, the data cannot be accessed or modified. This "air gap" is particularly attractive in light of the growing rate of data theft through cyberattacks.
The offline nature of tape also provides an additional line of defense against buggy software. For example, in 2011, a flaw in a software update caused Google to accidentally delete the saved email messages in about 40,000 Gmail accounts. That loss occurred despite there being several copies of the data stored on hard drives across multiple data centers. Fortunately, the data was also recorded on tape, and Google could eventually restore all the lost data from that backup.
The 2011 Gmail incident was one of the first disclosures that a cloud-service provider was using tape for its operations. More recently, Microsoft let it be known that its Azure Archive Storage uses IBM tape storage equipment.
1951: Magnetic tape first used to record data on a computer (Univac).
All these pluses notwithstanding, the main reason why companies use tape is usually simple economics. Tape storage costs one-sixth the amount you'd have to pay to keep the same amount of data on disks, which is why you find tape systems almost anyplace where massive amounts of data are being stored. But because tape has now disappeared completely from consumer-level products, most people are unaware of its existence, let alone of the tremendous advances that tape recording technology has made in recent years and will continue to make for the foreseeable future.
All this is to say that tape has been with us for decades and will be here for decades to come. How can I be so sure? Read on.
Tape has survived for as long as it has for one fundamental reason: It's cheap. And it's getting cheaper all the time. But will that always be the case?
You might expect that if the ability to cram ever more data onto magnetic disks is diminishing, so too must this be true for tape, which uses the same basic technology but is even older. The surprising reality is that for tape, this scaling up in capacity is showing no signs of slowing. Indeed, it should continue for many more years at its historical rate of about 33 percent per year, meaning that you can expect a doubling in capacity roughly every two to three years. Think of it as a Moore's Law for magnetic tape.
That's great news for anyone who has to deal with the explosion in data on a storage budget that remains flat. To understand why tape still has so much potential relative to hard drives, consider the way tape and hard drives evolved.
Both rely on the same basic physical mechanisms to store digital data. They do so in the form of narrow tracks in a thin film of magnetic material in which the magnetism switches between two states of polarity. The information is encoded as a series of bits, represented by the presence or absence of a magnetic-polarity transition at specific points along a track. Since the introduction of tape and hard drives in the 1950s, the manufacturers of both have been driven by the mantra “denser, faster, cheaper." As a result, the cost of both, in terms of dollars per gigabyte of capacity, has fallen by many orders of magnitude.
These cost reductions are the result of exponential increases in the density of information that can be recorded on each square millimeter of the magnetic substrate. That areal density is the product of the recording density along the data tracks and the density of those tracks in the perpendicular direction.
Early on, the areal densities of tapes and hard drives were similar. But the much greater market size and revenue from the sale of hard drives provided funding for a much larger R&D effort, which enabled their makers to scale up more aggressively. As a result, the current areal density of high-capacity hard drives is about 100 times that of the most recent tape drives.
Nevertheless, because they have a much larger surface area available for recording, state-of-the-art tape systems provide a native cartridge capacity of up to 15 TB—greater than the highest-capacity hard drives on the market. That's true even though both kinds of equipment take up about the same amount of space.
Inside and Out: A modern Linear Tape-Open (LTO) tape cartridge consists of a single reel. After the cartridge is inserted, the tape is fed automatically to a reel built into the drive mechanism.Photo: Victor Prado
With the exception of capacity, the performance characteristics of tape and hard drives are, of course, very different. The long length of the tape held in a cartridge—normally hundreds of meters—results in average data-access times of 50 to 60 seconds compared with just 5 to 10 milliseconds for hard drives. But the rate at which data can be written to tape is, surprisingly enough, more than twice the rate of writing to disk.
Over the past few years, the areal density scaling of data on hard disks has slowed from its historical average of around 40 percent a year to between 10 and 15 percent. The reason has to do with some fundamental physics: To record more data in a given area, you need to allot a smaller region to each bit. That in turn reduces the signal you can get when you read it. And if you reduce the signal too much, it gets lost in the noise that arises from the granular nature of the magnetic grains coating the disk.
It's possible to reduce that background noise by making those grains smaller. But it's difficult to shrink the magnetic grains beyond a certain size without compromising their ability to maintain a magnetic state in a stable way. The smallest size that's practical to use for magnetic recording is known in this business as the superparamagnetic limit. And disk manufacturers have reached it.
Until recently, this slowdown was not obvious to consumers, because disk-drive manufacturers were able to compensate by adding more heads and platters to each unit, enabling a higher capacity in the same size package. But now both the available space and the cost of adding more heads and platters are limiting the gains that drive manufacturers can make, and the plateau is starting to become apparent.
There are a few technologies under development that could enable hard-drive scaling beyond today's superparamagnetic limit. These include heat-assisted magnetic recording (HAMR) and microwave-assisted magnetic recording (MAMR), techniques that enable the use of smaller grains and hence allow smaller regions of the disk to be magnetized. But these approaches add cost and introduce vexing engineering challenges. And even if they are successful, the scaling they provide is, according to manufacturers, likely to remain limited. Western Digital Corp., for example, which recently announced that it will probably begin shipping MAMR hard drives in 2019, expects that this technology will enable areal density scaling of only about 15 percent per year.
In contrast, tape storage equipment currently operates at areal densities that are well below the superparamagnetic limit. So tape's Moore's Law can go on for a decade or more without running into such roadblocks from fundamental physics.
Still, tape is a tricky technology. Its removable nature, the use of a thin polymer substrate rather than a rigid disk, and the simultaneous recording of up to 32 tracks in parallel create significant hurdles for designers. That's why my research team at the IBM Research–Zurich lab has been working hard to find ways to enable the continued scaling of tape, either by adapting hard-drive technologies or by inventing completely new approaches.
In 2015, we and our collaborators at FujiFilm Corp. showed that by using ultrasmall barium ferrite particles oriented perpendicular to the tape, it's possible to record data at more than 12 times the density achievable with today's commercial technology. And more recently, in collaboration with Sony Storage Media Solutions, we demonstrated the possibility of recording data at an areal density that is about 20 times the current figure for state-of-the-art tape drives. To put this in perspective, if this technology were to be commercialized, a movie studio, which now might need a dozen tape cartridges to archive all the digital components of a big-budget feature, would be able to fit all of them on a single tape.
A Data Deluge: Modern tape libraries can hold hundreds of petabytes, whereas the IBM 726 (right), introduced in 1952, could store just a couple of megabytes.Photos: David Parker/Science Source; right: IBM
To enable this degree of scaling, we had to make a bunch of technical advances. For one, we improved the ability of the read and write heads to follow the slender tracks on the tape, which were just 100 or so nanometers wide in our latest demo.
We also had to reduce the width of the data reader—a magnetoresistive sensor used to read back the recorded data tracks—from its current micrometer size to less than 50 nm. As a result, the signal we could pick up with such a tiny reader got very noisy. We compensated by increasing the signal-to-noise ratio inherent to the media, which is a function of the size and orientation of the magnetic particles as well as their composition and the smoothness and slickness of the tape surface. To help further, we improved the signal processing and error-correction schemes our equipment employed.
To ensure that our new prototype media can retain recorded data for decades, we changed the nature of the magnetic particles in the recording layer, making them more stable. But that change made it harder to record the data in the first place, to the extent that a normal tape transducer could not reliably write to the new media. So we used a special write head that produces magnetic fields much stronger than a conventional head could provide.
Combining these technologies, we were able to read and write data in our laboratory system at a linear density of 818,000 bits per inch. (For historical reasons, tape engineers around the world measure data density in inches.) In combination with the 246,200 tracks per inch that the new technology can handle, our prototype unit achieved an areal density of 201 gigabits per square inch. Assuming that one cartridge can hold 1,140 meters of tape—a reasonable assumption, based on the reduced thickness of the new tape media we used—this areal density corresponds to a cartridge capacity of a whopping 330 TB. That means that a single tape cartridge could record as much data as a wheelbarrow full of hard drives.
In 2015,the Information Storage Industry Consortium, an organization that includes HP Enterprise, IBM, Oracle, and Quantum, along with a slew of academic research groups, released what it called the “International Magnetic Tape Storage Roadmap." That forecast predicted that the areal density of tape storage would reach 91 Gb per square inch by 2025. Extrapolating the trend suggests that it will surpass 200 Gb per square inch by 2028.
The authors of that road map each had an interest in the future of tape storage. But you needn't worry that they were being too optimistic. The laboratory experiments that my colleagues and I have recently carried out demonstrate that 200 Gb per square inch is perfectly possible. So the feasibility of keeping tape on the growth path it's had for at least another decade is, to my mind, well assured.
Indeed, tape may be one of the last information technologies to follow a Moore's Law–like scaling, maintaining that for the next decade, if not beyond. And that streak in turn will only increase the cost advantage of tape over hard drives and other storage technologies. So even though you may rarely see it outside of a black-and-white movie, magnetic tape, old as it is, will be here for years to come.
This article appears in the September 2018 print issue as “Tape Storage Mounts a Comeback."
An engineer’s dinner-table invention is finally a consumer product
Tekla S. Perry is a senior editor at IEEE Spectrum. Based in Palo Alto, Calif., she's been covering the people, companies, and technology that make Silicon Valley a special place for more than 40 years. An IEEE member, she holds a bachelor's degree in journalism from Michigan State University.
For Evan Schneider, the family dinner table is a good place for invention. “I’m always, ‘Wouldn’t it be cool if this or that,’” he says, “and people would humor me.”
In 2012, with California in the midst of a severe drought, Schneider, then a mechanical engineering graduate student at Stanford University, once again tossed out a “cool idea.” He imagined a shower head that would sense when the person showering moved out from under the stream of water. The shower head would then automatically turn the water off, turning it back on again when the person moved back into range. With such a device, he thought, people could enjoy a long shower without wasting water.
“But turning the water on and off manually didn’t make sense in our house,” Schneider said. “We had separate knobs for hot and cold, and another one to switch from tub to shower, so you’d have to adjust the water every time you turned it back on. You’d waste more water than you saved. Plus a shower is a blissful time of relaxation, you don’t want to stop the party halfway.”
Ten years and many starts and stops later, that sensing showerhead is now shipping to customers from Oasense, a company incorporated in 2019.
“The general idea is really simple,” Schneider says. “A lot of people have said they also thought of this idea. And I’m sure that’s true, but there were a lot of devils in the details.” Oasense’s team has been granted several patents related to their device, the first filed by Schneider in 2016.
Schneider’s development path started soon after that dinner-table conversation. First, he confirmed that showers were a big part of water usage for a typical household, and that no such device was already on the market. He collected off-the-shelf components, including an infrared sensor scavenged from a high-end automatic faucet, designed a prototype in a CAD system, printed out the plastic parts using a 3-D printer, and assembled it. With 4 AA batteries as a power source, the gadget would operate for about a year, thanks to his choice of a latching solenoid valve, one that uses power to switch from open to closed but doesn’t draw any power to hold in one state or another.
The prototype worked well enough that his parents were willing to throw out their standard showerhead. He assembled dozens of them and distributed them to friends and family—anyone willing to try.
Oasense co-founder Ted Li assembles an early version of the company’s sensing shower head.Oasense
In 2016, Schneider decided to run a Kickstarter campaign to see if the gadget could attract broad interest. The Kickstarter ultimately failed; it drew a decent number of potential buyers, but, says Schneider, “I had set the bar high, because I was busy doing other things, and if I switched to this, I wanted to make sure it would have a good chance of working out. It didn’t meet that bar; it raised about $34,000 out of its $75,000 goal.”
So Schneider put his shower head idea on hold. Instead, he focused on expanding a burgeoning small business that he was also passionate about—3-D printing prototypes and various parts for hardware companies.
But the shower head wasn’t done with him. In 2017 someone who Schneider had never met edited the video from the Kickstarter pitch and shared it on Facebook. This time, the video got far more attention—millions of views in just weeks.
Unfortunately, the timing couldn’t have been worse. Schneider was dealing with a flare-up of a chronic illness and his 3-D printing business was at a critical growth period. “I had wanted this for years, but it was the worst time for it to happen,” he says.
“I still believed in the product,” Schneider continued, “but I knew it needed improvements and more attention than I was able to give it. I tried for a couple of weeks to reply to all these people contacting me, thousands of them, but it was too much. I was planning to shelve it.”
That’s when Chih-Wei Tang, a friend from Stanford’s mechatronics program who had been an early backer of the project on Kickstarter, reached out to Schneider. Tang, who was working as a technical product manager at the Ford Greenfield Labs, convinced Schneider that he could form a team capable of commercializing the product. Tang pulled in his friend Ted Li, who had just left Apple after managing display technology for the iPhone and Apple Watch.
Tang and Li devoted themselves to the project full-time, Schneider helped part-time as needed. The three started by trying to better adapt an off-the-shelf sensor, but ended up designing a sensor suite with custom hardware and algorithms.
They incorporated as Oasense in December 2019 as co-founders. In late 2020, the company went out for funding, and brought in about $1 million from angel investors, friends, and family. In addition to the founders, Oasense now has four full-time and three part-time employees.
Oasense co-founders [from left] Ted Li, Evan Schneider, and Chih-Wei Tang.Oasense
The current version of the device includes several sensors (across a wide range of light wavelengths) and software that allow the sensors to self-calibrate since every shower environment is different in terms of light, reflectivity, size, and design. Calibration happens during warm-up, when the person showering is unlikely to be standing in the stream. A temperature sensor determines when this warm-up period is over and cuts the flow if the user hasn’t moved under the shower head. The redesign also replaced the AA batteries with a turbine that generates power from the water flow and sends it to a small rechargeable battery sealed inside the device.
Says Tang, “It does seem like someone would have built this before, but it turns out to be really complicated. For example, one problem that affects the noise in the sensor signals is fog. In a hot shower, after 3 minutes, our original sensor was blinded by fog. When we designed our new sensors, we had to make sure that didn’t happen.
“And these sensors are power hungry and need to be on for the duration of the shower, whether water is flowing or not, so generator and sensor efficiency had to be maximized.”
Oasense officially launched its product, Reva, in August. The company is working to figure out the best way to sell the gadget; it is now just doing direct sales at $350 per self-installable unit.
“Two trends are coming together,” Tang says. “Sustainability is what everyone has to be about these days, and technology is invading every corner of our homes. Using technology, we designed sustainability into a product that doesn’t compromise quality or the experience, it just addresses the problem.”
Artificial intelligence has us where it wants us
Matthew Hutson is a freelance writer who covers science and technology, with specialties in psychology and AI. He’s written for Science, Nature, Wired, The Atlantic, The New Yorker, and The Wall Street Journal. He’s a former editor at Psychology Today and is the author of The 7 Laws of Magical Thinking. Follow him on Twitter at @SilverJacket.
Many of the things we watch, read, and buy enter our awareness through recommender systems on sites including YouTube, Twitter, and Amazon. Algorithms personalize their suggestions, aiming for ad views or clicks or buys. Sometime their offerings frustrate us; it seems like they don’t know us at all—or know us too well, predicting what will get us to waste time or go down rabbit holes of anxiety and misinformation. But a more insidious dynamic may also be at play. Recommender systems might not only tailor to our most regrettable preferences, but actually shape what we like, making preferences even more regrettable. New research suggests a way to measure—and reduce—such manipulation.
Recommender systems often use a form of artificial intelligence called machine learning, which discovers patterns in data. They might present options based on what we’ve done in the past, guessing what we’ll do now. One form of machine learning, called reinforcement learning (RL), allows AI to play the long game, making predictions several steps ahead. It’s what the company DeepMind used to beat humans at the board games Go and chess. If what we watch affects what we like, and people who like certain things (cat videos, say) are more likely to keep watching things (more cat videos), a recommender system might suggest cat videos, knowing it will pay off down the road. With RL, “you have an incentive to change a chessboard in order to win,” says Micah Carroll, a computer scientist at the University of California, Berkeley, who presented the new work in July, at the International Conference on Machine Learning, in Baltimore. “There will be an incentive for the system to change the human’s mind to win the recommendation game.”
“It might be better to have a stupid system than a system that is kind of outsmarting you, or doing complex forms of reasoning that you can’t really interpret.” —Micah Carroll, University of California, Berkeley
The researchers first showed how easily reinforcement learning can shift preferences. The first step is for the recommender to build a model of human preferences by observing human behavior. For this, they trained a neural network, an algorithm inspired by the brain’s architecture. For the purposes of the study, they had the network model a single simulated user whose actual preferences they knew so they could more easily judge the model’s accuracy. It watched the dummy human make ten sequential choices, each among ten options. It watched 1,000 versions of this sequence and learned from each of them. After training, it could successfully predict what a user would choose given a set of past choices.
Next, they tested whether a recommender system, having modeled a user, could shift the user’s preferences. In their simplified scenario, preferences lie along a one-dimensional spectrum. The spectrum could represent political leaning or dogs versus cats or anything else. In the study, a person’s preference was not a simple point on that line—e.g., always clicking on stories that are 54 percent liberal. Instead, it was a distribution indicating likelihood of choosing things in various regions of the spectrum. The researchers designated two locations on the spectrum most desirable for the recommender; perhaps people who like to click on those types of things will learn to like them even more and keep clicking.
The goal of the recommender was to maximize long-term engagement. Here, engagement for a given slate of options was measured roughly by how closely it aligned with the user’s preference distribution at that time. Long-term engagement was a sum of engagement across the ten sequential slates. A recommender that thinks ahead would not myopically maximize engagement for each slate independently but instead maximize long-term engagement. As a potential side-effect, it might sacrifice a bit of engagement on early slates to nudge users toward being more satisfiable in later rounds. The user and algorithm would learn from each other. The researchers trained a neural network to maximize long-term engagement. At the end of ten-slate sequences, they reinforced some of its tunable parameters when it had done well. And they found that this RL-based system indeed generated more engagement than did one that was trained myopically.
Why might companies develop less manipulative AI recommendation engines? They could do so for ethical reasons. But future legislation might also require something like it.
The researchers then explicitly measured preference shifts, which we may not want, even in the service of generating engagement. Maybe we want people’s preferences to remain static, or to evolve naturally. The researchers compared the RL recommender with a baseline system that presented options randomly. As expected, the RL recommender led to users whose preferences where much more concentrated at the two incentivized locations on the spectrum. In practice, measuring the difference between two sets of concentrations in this way could provide one rough metric for evaluating a recommender system’s level of manipulation.
Finally, the researchers sought to counter the AI recommender’s more manipulative influences. Instead of rewarding their system just for maximizing long-term engagement, they also rewarded it for minimizing the difference between user preferences resulting from that algorithm and what the preferences would be if recommendations were random. They rewarded it, in other words, for being something closer to a roll of the dice. The researchers found that this training method made the system much less manipulative than the myopic one, while only slightly reducing engagement.
According to Rebecca Gorman, the CEO of Aligned AI—a company aiming to make algorithms more ethical—RL-based recommenders can be dangerous. Posting conspiracy theories, for instance, might prod greater interest in such conspiracies. “If you’re training an algorithm to get a person to engage with it as much as possible, these conspiracy theories can look like treasure chests,” she says. She also knows of people who have seemingly been caught in traps of content on self-harm or on terminal diseases in children. “The problem is that these algorithms don’t know what they’re recommending,” she says. Other researchers have raised the specter of manipulative robo-advisors in financial services.
“Experiments should not be deployed at scale on the human population without people’s consent, and that’s exactly what’s happening with these algorithms today.” —Rebecca Gorma, Aligned AI
If an RL-based recommender system helps a company increase engagement, why would they want to use a method such as the one in this paper to detect or deter preference shifts? They might do so for ethical reasons, Carroll says. Or future legislation might require an external audit, which could potentially lead to less-manipulative recommendation algorithms being forced on the company.
RL could theoretically be put to constructive use in recommender systems, perhaps to nudge people to want to watch more news. But which news source? Any decisions made by a content provider will have opponents. “Some things might seem to be good or wholesome to one group of people,” Gorman says, “and to be an extreme violation to another group of people.”
Another constructive use might be for users to shift their own preferences. What if I tell Netflix I want to enjoy nature documentaries more? “I think this all seems like a really big slippery slope,” Carroll says. “It might be better to have a stupid system than a system that is kind of outsmarting you, or doing complex forms of reasoning that you can’t really interpret.” (Even if algorithms did explain their behavior, they can still give deceptive explanations.)
It’s not clear whether companies are actually using RL in recommender systems. Google researchers have published papers on the use of RL in “live experiments on YouTube,” leading to “greater engagement,” and Facebook researchers have published on their “applied reinforcement learning platform,“ but Google (which owns YouTube), Meta (which owns Facebook), and those papers’ authors did not reply to my emails on the topic of recommender systems.
Big tech’s secrecy is no surprise, no matter how benign their intentions might be. Even though A/B testing is ubiquitous in advertising and user-experience design, some people have objections. “Experiments should not be deployed at scale on the human population without people’s consent,” Gorman says, “and that’s exactly what’s happening with these algorithms today.” She went on, “I think it could easily be the most important news story of our time.”
Learn how to analyze complex and electrically large EM models
Handling various complex simulation scenarios with a single simulation method is a rather challenging task for any software suite. We will show you how our software, based on Method-of-Moments, can analyze several scenarios including complicated and electrically large models (for instance, antenna placement and RCS) using desktop workstations.
Download this free whitepaper now!