How come search engines don't crawl the list and I can't find 'robots.txt'?

• To: Georgi Guninski <[email protected]>
• Subject: How come search engines don't crawl the list and I can't find 'robots.txt'?
• From: [email protected] (coderman)
• Date: Sun, 13 Apr 2014 07:08:27 -0700
• Cc: cpunks <[email protected]>
• In-reply-to: <[email protected]$>
• References: <[email protected]$>

On Sun, Apr 13, 2014 at 6:53 AM, Georgi Guninski <[email protected]> wrote:
> AFAICT the list archives don't show in search engines
> and in addition don't see /robots.txt.
>
> How so?

back when traffic was handled on the al-qaeda.net domain, indexing by
western search engines was considered to be "material support to
Al-qaeda"...

now they've just forgotten to undo their filters.

;)

• Follow-Ups:
  • How come search engines don't crawl the list and I can't find 'robots.txt'?
    • From: [email protected] (Georgi Guninski)
  • How come search engines don't crawl the list and I can't find 'robots.txt'?
    • From: [email protected] (Linux User)

• References:
  • How come search engines don't crawl the list and I can't find 'robots.txt'?
    • From: [email protected] (Georgi Guninski)

• Prev by Date: NTRU Prime implementation
• Next by Date: [cryptome] Re: snowman news
• Previous by thread: How come search engines don't crawl the list and I can't find 'robots.txt'?
• Next by thread: How come search engines don't crawl the list and I can't find 'robots.txt'?
• Index(es):
  • Date
  • Thread