Archive.org is getting shitter, then

Open discussion about any topic, as long as you abide by the rules of course!
Post Reply
User avatar
Foo
Posts: 13840
Joined: Thu Aug 03, 2000 7:00 am
Location: New Zealand

Archive.org is getting shitter, then

Post by Foo »

Just tried to look up http://www.mdqnet.net, which maybe a very small number of you would remember was a major Q1 mod development centre back in the day.

Up until mebbe 6 months ago, archive.org would list that site no problem, I could scroll back and see it in (a few of) its original states.

Now, archive.org reports that site is blocked by robots.txt. Problem is, the domain expired about 3 years ago and it's just a link farm domain now, owned by someone completely different.

So archive.org looks like it'll be come practically useless from now on then. I can't think of many sites which, when they expire, don't fall into the hands of link farmers... and if they're blocking with robots.txt, and archive.org is making that apply to the whole history of the site... well... shit.
"Maybe you have some bird ideas. Maybe that’s the best you can do."
― Terry A. Davis
User avatar
Zerofactor
Posts: 890
Joined: Thu May 31, 2001 7:00 am

Post by Zerofactor »

Post Reply