Removal from Archive.org

I like the Wayback Machine. It has saved me multiple times when researching an issue only to find that the website is defunct. But, after deprecating my old site, silverjaguar.com, I wasn’t sure if I wanted to have my old posts stay up there. Perhaps if it was a complete archive, I might have left it there. Since it wasn’t, I decided to remove it.

I did review all the pages that were stored and grabbed some of my old blog posts (which I’ll eventually migrate here). It was interesting to see how I changed as a developer and a person. Sometimes, it was embarrassing. Other times, I wondered what happened to that person and if they’ll return.

The steps to remove were reasonably straightforward. Fortunately, I still own the domain. If I hadn’t, I’m not sure how much of a pain this might have been.

First, I removed the domain redirect to my LinkedIn page and set up a small static website. Since the Wayback Machine archived non-SSL versions of my site, I ensured that the http:// version was accessible. I also enabled the https://, http://www, and https://www versions for good measure.

Second, I created a robots.txt file:

User-agent: *
Disallow: /

User-agent: archive.org_bot
Disallow: /

User-agent: ia_archiver
Disallow: /

The first section requests that all search engines not index the page. I added the second two sections because I had seen that on other sites. It may have been redundant, but I wasn’t sure if an automated or manual process would specifically look for it.

Finally, I sent a short email to info@archive.org asking that they remove my site:

To: info@archive.org

Subject: Removal of silverjaguar.com

Greeting! I would like to remove the archived content for silverjaguar.com and its associated pages. There is a robots.txt file (http://silverjaguar.com/robots.txt) that should prevent crawls/indexing.

I thank you in advance! Please let me know if there is anything you need from me.

The next business day, I received the following response:

The following has been submitted for exclusion from the Wayback Machine at web.archive.org:

silverjaguar.com

Please allow up to a day for the automated portions of the process to run their course and for the changes to take effect.

One day later, and now when I search, I see, “This URL has been excluded from the Wayback Machine.” The process was much more straightforward than I expected.

Of course, this is a good reminder that things posted on the internet tend to stay there. I wonder what I’ll think of my present self another twenty years from now.