How Pixar recovered Toy Story 2 after a Unix command deleted nearly the entire film in 1998

Skye Jacobs

Posts: 2,012   +59
Staff
Oops: Twenty-eight years ago, Pixar nearly lost 90% of Toy Story 2's digital files – not because a system crashed, but because someone ran a routine Unix command that engineers had been using for years without a second thought. The command /bin/rm -r -f instructs the system to recursively delete everything under a directory without asking for confirmation. At Pixar in 1998, it was apparently executed in the wrong location.

The studio's animation pipeline at the time ran across a network of Unix and Linux machines holding hundreds of thousands of production files. Artists and technical staff had broad access to both personal workspaces and shared production directories. The setup made collaboration easier, but meant a routine cleanup command could reach critical files if issued from the wrong directory.

According to people familiar with the incident, the command propagated beyond its intended scope and began erasing core production data. Oren Jacob, an associate technical director on the film, watched it happen in real time. Files began disappearing from his screen, first individual assets, then entire characters and sequences.

Within moments, roughly 90% of the movie was gone.

"You don't often watch a company vaporize in front of your eyes," Jacob told The Wall Street Journal.

The immediate response was to contain the damage. An emergency call went out to shut down the system, cutting off the deletion mid-process. That worked. The fallback plan did not.

Pixar's backup system was designed precisely for this kind of scenario, but it had been failing silently. Nobody realized it until they tried to use it. "The mechanism we had in place specifically to help us recover from data failures had itself failed," Pixar co-founder Ed Catmull later wrote in Creativity, Inc.

What remained of the film was fragmented and inconsistent. Reconstructing it from pieces would have been slow and uncertain, and the timeline was already tight. Pixar had recently committed to releasing Toy Story 2 theatrically rather than as a direct-to-video sequel, raising both the creative bar and the financial stakes. A long delay could have had serious consequences for the company.

To Pixar's leadership, identifying who triggered the error was beside the point. "Looking for someone to blame doesn't help us learn from mistakes," Catmull said. "We understood that the deletion of the movie was an accident because somebody typed in a command when they were in the wrong directory. We don't know who typed the command, or if they even knew that they were the one who did it, but it didn't matter." What mattered was whether the film could be recovered at all.

The answer turned out to be sitting off-site. Galyn Susman, the film's supervising technical director, had been maintaining a working copy of the project on a machine at her home. She had built a remote workflow during her pregnancy, periodically syncing updated versions of the film so she could continue working after hours. It wasn't part of Pixar's formal backup infrastructure, but it was current enough to matter.

That machine held what was likely the only intact version of the film. Susman and Jacob went to retrieve it. The computer was carefully packed into a car and driven back to Pixar's campus. According to those familiar with the trip, they took no chances with the hardware, treating it as if it were fragile infrastructure rather than a personal workstation.

Once powered up inside the studio, the system delivered what the internal backups could not: a usable copy of the movie.

Recovery still wasn't instantaneous. Teams spent days combing through tens of thousands of files, checking for corruption, missing assets, and inconsistencies. Given how tightly coupled the animation pipeline had become, the process was as much verification as restoration.

The incident exposed several now-obvious gaps: broad access to critical directories, backups that were never tested under real conditions, and a single central store of production data with no distributed redundancy. Modern pipelines use version control, automated backup testing, and distributed storage to make it far harder for one errant command to take out an entire production. In 1998, those safeguards were either incomplete or absent.

Despite the close call, Pixar didn't simply finish and ship the film as it was. In the final year before release, Toy Story 2 underwent extensive creative overhaul. Catmull later described it as "the cinematic equivalent of a heart transplant."

Susman stayed closely tied to the project through completion. "Personally," she said, "I still think Toy Story 2 is the best of the franchise."

Her son, Eli, appears in the film's credits as part of Pixar's "production babies" tradition, which recognizes children born during a movie's development. In this case, the connection runs deeper. The same home setup built around that period ended up preserving the film itself.

Permalink to story:

 
Maybe rm was different back then, but the target file/directory is necessary now. Running rm -rf / will delete all your files (-rf is the same thing as -r -f since single-dash options are single character options and can be combined). Deleting the current directory would instead use a dot (.) or .. for the parent directory, although it is not allowed today and * is necessary instead like so: rm -rf *
 
Maybe rm was different back then, but the target file/directory is necessary now. Running rm -rf / will delete all your files (-rf is the same thing as -r -f since single-dash options are single character options and can be combined). Deleting the current directory would instead use a dot (.) or .. for the parent directory, although it is not allowed today and * is necessary instead like so: rm -rf *

It worked the same back then. The original article in the WSJ correctly stated it as you did:

/bin/rm -r -f *

which can obviously be compressed to rm -rf *

The command also can (reasonably obviously) be entirely benign.

prompt#> cd /tmp/testdir
prompt#> ls
prompt#> file1 file2 file3
prompt#> rm -rf *
prompt#> ls
prompt#>

What matters is what directory you are in at the time that you run it.
 
Honestly the most underrated detail is that they physically drove the machine back to the studio instead of just pulling it over the network. Either the connection was too slow, or someone was deeply, deeply paranoid about that single copy. Smart call either way.
 
Honestly the most underrated detail is that they physically drove the machine back to the studio instead of just pulling it over the network. Either the connection was too slow, or someone was deeply, deeply paranoid about that single copy. Smart call either way.

It was 1998. Even if they had a T1 - at both the staffer's home and at pixar, at 45mbps, it would have taken weeks to months to transfer it.
 
To think that nowadays you could render the complete Toy Story 2 movie in a matter of minutes even with better IQ.
 
It could have been may have been intentional. Do you like how I parsed my statement ? Battery life could increase by 1000%
 
To think that nowadays you could render the complete Toy Story 2 movie in a matter of minutes even with better IQ.

That is an interesting question - here's what a query had to say about it (I will apologize for the long text here in advance!):

Rendering Toy Story 2 (1999) on a single average modern computer involves a trade-off between visual fidelity and rendering method. While the original production took months using a massive render farm, a modern home PC could replicate the result in weeks or achieve a similar look in real-time depending on the approach.

The original Toy Story 2 was significantly more complex than its predecessor.

  • Render Time: Frames took an average of 20–30 minutes to render on 1999 hardware, with complex scenes reaching 40+ minutes.
  • Total Effort: The film required roughly 800,000 machine hours across hundreds of servers.
  • Frame Count: At 24 frames per second and a runtime of ~92 minutes, the film contains approximately 132,480 frames.
  • Modern Single-PC Estimates:
The time required today depends entirely on whether you are using modern rendering engines (which use shortcuts like denoising and GPU acceleration) or attempting to simulate the original CPU-based ray tracing algorithms.

1. High-Fidelity Replica (Original Algorithms)
If you attempted to render the movie using the exact same mathematical methods as 1999 (pure CPU ray tracing without modern optimizations) on a standard modern processor:

Speedup Factor: Modern CPUs are roughly 1,000x to 2,000x faster than the 1995-1999 era hardware used for the original.
Estimated Time: The average frame time would drop from ~25 minutes to roughly 1–2 minutes per frame.
Total Duration: Rendering the entire film sequentially on one machine would take approximately 90 to 180 days (3–6 months).
2. Modern Equivalent Quality (GPU Rendering)
If you used modern software (like Blender Cycles, Unreal Engine, or Pixar's own RenderMan on a GPU) to achieve a visually similar result using current technology:

Estimated Time: A high-end consumer GPU (e.g., RTX 40-series) could render a frame of that complexity in 10–30 seconds or less.
Total Duration: The full movie could be rendered in 15 to 45 days on a single powerful card.
Real-Time Potential: With game-engine technology (rasterization + ray tracing), scenes of this complexity can often be rendered in real-time (30–60 fps), meaning the whole movie could theoretically be "rendered" in the time it takes to watch it (~92 minutes), though this requires pre-baking lighting data which takes additional time.

Key Technical Constraints:
Software Obsolescence: The original render files rely on proprietary software from the 90s that will not run on modern operating systems without significant porting effort, as noted by Pixar staff during the 3D re-release conversion.
Memory Bottlenecks: While modern GPUs are faster, they are often memory-constrained compared to the massive RAM pools of a render farm. Complex scenes from the original might need to be broken down further or optimized to fit on a single consumer card.
Algorithmic Efficiency: Modern renderers use "denoisers" that allow them to use fewer light samples (rays) per pixel while maintaining a clean image, drastically cutting render times compared to the brute-force methods of 1999

So... it could take much longer than a few minutes OR it could indeed be rendered in real-time, depending on the approach taken and the quality of the output. How far we've come, in some regards! :)
 
Last edited:
That is an interesting question - here's what a query had to say about it (I will apologize for the long text here in advance!):

Rendering Toy Story 2 (1999) on a single average modern computer involves a trade-off between visual fidelity and rendering method. While the original production took months using a massive render farm, a modern home PC could replicate the result in weeks or achieve a similar look in real-time depending on the approach.

The original Toy Story 2 was significantly more complex than its predecessor.

  • Render Time: Frames took an average of 20–30 minutes to render on 1999 hardware, with complex scenes reaching 40+ minutes.
  • Total Effort: The film required roughly 800,000 machine hours across hundreds of servers.
  • Frame Count: At 24 frames per second and a runtime of ~92 minutes, the film contains approximately 132,480 frames.
  • Modern Single-PC Estimates:
The time required today depends entirely on whether you are using modern rendering engines (which use shortcuts like denoising and GPU acceleration) or attempting to simulate the original CPU-based ray tracing algorithms.

1. High-Fidelity Replica (Original Algorithms)
If you attempted to render the movie using the exact same mathematical methods as 1999 (pure CPU ray tracing without modern optimizations) on a standard modern processor:

Speedup Factor: Modern CPUs are roughly 1,000x to 2,000x faster than the 1995-1999 era hardware used for the original.
Estimated Time: The average frame time would drop from ~25 minutes to roughly 1–2 minutes per frame.
Total Duration: Rendering the entire film sequentially on one machine would take approximately 90 to 180 days (3–6 months).
2. Modern Equivalent Quality (GPU Rendering)
If you used modern software (like Blender Cycles, Unreal Engine, or Pixar's own RenderMan on a GPU) to achieve a visually similar result using current technology:

Estimated Time: A high-end consumer GPU (e.g., RTX 40-series) could render a frame of that complexity in 10–30 seconds or less.
Total Duration: The full movie could be rendered in 15 to 45 days on a single powerful card.
Real-Time Potential: With game-engine technology (rasterization + ray tracing), scenes of this complexity can often be rendered in real-time (30–60 fps), meaning the whole movie could theoretically be "rendered" in the time it takes to watch it (~92 minutes), though this requires pre-baking lighting data which takes additional time.

Key Technical Constraints:
Software Obsolescence: The original render files rely on proprietary software from the 90s that will not run on modern operating systems without significant porting effort, as noted by Pixar staff during the 3D re-release conversion.
Memory Bottlenecks: While modern GPUs are faster, they are often memory-constrained compared to the massive RAM pools of a render farm. Complex scenes from the original might need to be broken down further or optimized to fit on a single consumer card.
Algorithmic Efficiency: Modern renderers use "denoisers" that allow them to use fewer light samples (rays) per pixel while maintaining a clean image, drastically cutting render times compared to the brute-force methods of 1999

So... it could take much longer than a few minutes OR it could indeed be rendered in real-time, depending on the approach taken and the quality of the output. How far we've come, in some regards! :)
I should just be impressed by the improvements in rendering speed but actually I'm astounded by the AI that generated that explanation.
 
That is an interesting question - here's what a query had to say about it (I will apologize for the long text here in advance!):

Rendering Toy Story 2 (1999) on a single average modern computer involves a trade-off between visual fidelity and rendering method. While the original production took months using a massive render farm, a modern home PC could replicate the result in weeks or achieve a similar look in real-time depending on the approach.

The original Toy Story 2 was significantly more complex than its predecessor.

  • Render Time: Frames took an average of 20–30 minutes to render on 1999 hardware, with complex scenes reaching 40+ minutes.
  • Total Effort: The film required roughly 800,000 machine hours across hundreds of servers.
  • Frame Count: At 24 frames per second and a runtime of ~92 minutes, the film contains approximately 132,480 frames.
  • Modern Single-PC Estimates:
The time required today depends entirely on whether you are using modern rendering engines (which use shortcuts like denoising and GPU acceleration) or attempting to simulate the original CPU-based ray tracing algorithms.

1. High-Fidelity Replica (Original Algorithms)
If you attempted to render the movie using the exact same mathematical methods as 1999 (pure CPU ray tracing without modern optimizations) on a standard modern processor:

Speedup Factor: Modern CPUs are roughly 1,000x to 2,000x faster than the 1995-1999 era hardware used for the original.
Estimated Time: The average frame time would drop from ~25 minutes to roughly 1–2 minutes per frame.
Total Duration: Rendering the entire film sequentially on one machine would take approximately 90 to 180 days (3–6 months).
2. Modern Equivalent Quality (GPU Rendering)
If you used modern software (like Blender Cycles, Unreal Engine, or Pixar's own RenderMan on a GPU) to achieve a visually similar result using current technology:

Estimated Time: A high-end consumer GPU (e.g., RTX 40-series) could render a frame of that complexity in 10–30 seconds or less.
Total Duration: The full movie could be rendered in 15 to 45 days on a single powerful card.
Real-Time Potential: With game-engine technology (rasterization + ray tracing), scenes of this complexity can often be rendered in real-time (30–60 fps), meaning the whole movie could theoretically be "rendered" in the time it takes to watch it (~92 minutes), though this requires pre-baking lighting data which takes additional time.

Key Technical Constraints:
Software Obsolescence: The original render files rely on proprietary software from the 90s that will not run on modern operating systems without significant porting effort, as noted by Pixar staff during the 3D re-release conversion.
Memory Bottlenecks: While modern GPUs are faster, they are often memory-constrained compared to the massive RAM pools of a render farm. Complex scenes from the original might need to be broken down further or optimized to fit on a single consumer card.
Algorithmic Efficiency: Modern renderers use "denoisers" that allow them to use fewer light samples (rays) per pixel while maintaining a clean image, drastically cutting render times compared to the brute-force methods of 1999

So... it could take much longer than a few minutes OR it could indeed be rendered in real-time, depending on the approach taken and the quality of the output. How far we've come, in some regards! :)
If I'm not mistaken, back then Pïxar used Renderman, which I think nowadays is GPU accelerated. Back when SW EP1 was in production, nvidia previs systems rendered scenes in real time with IQ that was close to final rendering. Today scenes in shows like The Mandalorian are rendered in real time


So my guess is that a high end PC should be able to render Toy Story 2 in real time.
 
I should just be impressed by the improvements in rendering speed but actually I'm astounded by the AI that generated that explanation.

I don't know if that explanation is totally right tho.

The "1000x–2000x faster" section seems fishy

If the original average frame took about 25 minutes to render then:

25 min / 1000 = 1.5 seconds
25 min / 2000 = 0.75 seconds

So a true 1000x-2000x speedup would reduce the render time to around 1 second per frame, not 1-2 minutes per frame.

The 1-2 minute estimate would correspond to a speedup of only about 12x-25x.

I suspect the AI either mixed up CPU speedup with actual rendering speedup, or worked backward from the 90-180 day total render estimate and accidentally left the 1000x-2000x figure in the response.

The overall point still stands that modern hardware is vastly faster, but those particular numbers don't seem mathematically consistent with each other.
 
UNIX Permissions

For all file system objects, the access rights of the owner, the owner's group, and other subjects are set. The owner of the object can be any user of the system. A group can be any group in the system.

An object can only be owned by one user and one group.

When setting up, the following types of access rights are used:

r — the right to open an object for reading;

w — the right to open an object for writing;

x - the right to execute the object (for directories - the right to read the contents of the directory);

t - sticky bit, for directories it prohibits deleting a file in the directory for everyone except the owners;

SUID - the right to execute a file on behalf of its owner;

SGID - the right to execute a file on behalf of the owner's group (for directories - group inheritance for objects in the directory).

Only the owner or administrator can change the access rights of an object.

When an object is created, its owner becomes the subject who created the object. The object will also belong to the group of the subject who created it.

Only the administrator has the right to change the owner of an object.

Changing the owner group of an object is allowed to the owner of that object and the administrator.

-----------------------

Taking work materials home is strictly prohibited.
 
Honestly the most underrated detail is that they physically drove the machine back to the studio instead of just pulling it over the network. Either the connection was too slow, or someone was deeply, deeply paranoid about that single copy. Smart call either way.

Over the Internet, in the late 90's???
That would be like pulling teeth...
 
Yes — the arithmetic is badly tangled.

The biggest problem is that the figures are mixing different claims and then multiplying them as though they belong together.

At 92 minutes × 60 seconds × 24 fps, the film would contain about 132,480 frames. That part is fine.

But if each frame took 20–30 minutes, then rendering the whole film on one equivalent machine would take:

20 minutes per frame: 1,840 days, or about 5 years.
25 minutes per frame: 2,300 days, or about 6.3 years.
30 minutes per frame: 2,760 days, or about 7.6 years.

So 20–30 minutes per frame does not lead to 90–180 days. It leads to several years on one machine.

The 800,000 machine-hours figure is also suspect in that paragraph because that famous number is usually associated with the first Toy Story, not necessarily Toy Story 2. Wired reported that Toy Story used 117 Sun SPARC 20s and required about 800,000 machine-hours for the final cut; Britannica also gives Toy Story as 114,240 frames and 800,000 machine-hours.

But even taking 800,000 machine-hours and applying it to a 92-minute film, the average would be:

800,000 ÷ 132,480 = about 6 machine-hours per frame.

So the text cannot sensibly say both:

‘average frame time was 20–30 minutes’, and
‘total effort was 800,000 machine-hours’

Those are not remotely the same scale. 20–30 minutes per frame gives roughly 44,000–66,000 machine-hours, not 800,000.

The other glaring error is here:

Modern CPUs are roughly 1,000x to 2,000x faster... average frame time would drop from ~25 minutes to roughly 1–2 minutes per frame.

No. If you take 25 minutes and divide it by 1,000, you get 1.5 seconds, not 1–2 minutes.

If you divide it by 2,000, you get 0.75 seconds.

So that section is mathematically upside-down. A 1,000–2,000× speed-up from 25 minutes would make the full film something like 1–3 days, not 3–6 months, assuming the original 25-minute figure were valid.

If instead you use the 800,000 machine-hours figure and divide by a claimed 1,000–2,000× modern speed-up, the total becomes:

800 hours at 1,000× faster = about 33 days
400 hours at 2,000× faster = about 17 days

That is much closer to the GPU estimate than the text admits.

The GPU estimate is actually the most arithmetically coherent part. At 10–30 seconds per frame, the whole film would take about 15–46 days.

So the corrected gist is:

A single modern machine rendering a Toy Story 2-like film could plausibly take anything from days to weeks to months, depending on renderer, samples, resolution, lighting, denoising, GPU use, scene optimisation, and whether you are recreating the original pipeline or merely matching the visible look. But the quoted numbers do not support the conclusion as written. They mix per-frame wall time, machine-hours, render-farm parallelism, and modern speed-up in a way that makes the estimates mathematically inconsistent.
 
Taking work materials home is strictly prohibited.

Did you work at Pixar in 1998? Or do you have their employment policy document in hand and can quote from it?

Absent that, you're making a broad claim that's unsupportable. I can state without fear of contradiction that it's false, based on my experience as a UNIX/linux systems administrator from 1992 to 2022.

A company's policies, or lack thereof, are what determine the permissibility of taking work material home. Much of the global economy would have been devastated had such policies not been modified and adjusted during COVID.
 
But if I write that I worked at Pixar in 1998, will you believe me? Or do I need to attach some screenshots?
If I had worked there, then nothing like this would have happened.

You don't _have_ to do anything.

Meaningless hypotheticals don't advance your claims however.
 
Well, I can say I work at Google.
And that's probably true. But I could be a cleaner, a CEO, or a programmer.
Of course, it's unclear who's more useful, but the level of access to information varies.
Society is divided into informationally autonomous groups and the Internet will not change this.

I was doing mathematical modeling and not OS or database administration, but I still observed it myself.
 
Last edited:
Back