[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
How come search engines don't crawl the list and I can't find 'robots.txt'?
> Message du 13/04/14 17:07
> De : "Georgi Guninski"
> A : "coderman"
> Copie à : "cpunks"
> Objet : Re: How come search engines don't crawl the list and I can't find 'robots.txt'?
>
> On Sun, Apr 13, 2014 at 07:08:27AM -0700, coderman wrote:
> > On Sun, Apr 13, 2014 at 6:53 AM, Georgi Guninski wrote:
> > > AFAICT the list archives don't show in search engines
> > > and in addition don't see /robots.txt.
> > >
> > > How so?
> >
> > back when traffic was handled on the al-qaeda.net domain, indexing by
> > western search engines was considered to be "material support to
> > Al-qaeda"...
> >
> > now they've just forgotten to undo their filters.
> >
> > ;)
>
> lol...
>
> the russian and chinese comrades don't show the
> list too, might be for technical reasons.
>
> does a single search engine crawls
> the archives and displays the results?
>
I have heard about the existence of this mailing list for years ... yet never found it with search engines.
I never did a positive effort to find it too, let's be honest. It was kind of "one day I may meet that crowd by chance".
One day a few months ago I found a guy that posted in another list I was subscribed and also here. Lo and behold, that's the famed mailing list that cannot be found anywhere where reasonable people would search.
You guys are hidden in the deepwebz, Google doesn't index it, you are the bad bad turrists, lol.