[BBLISA] Help with .htaccess file

[Public]

On Mon, 2004-08-02 at 19:24, Scott Ehrlich wrote:
> I have my domain name hosted with pair.com, who uses apache 1.3.29 on
> freebsd boxes.
> 
> I want to either:
> 
> 1) Use a .htaccess file to prevent any web crawlers/robots from gaining
> access to one or more directories off my public_html folder
> 
> 2) Use SSL to do the same
> 
> I do not have access to the apache config file, so I presume the .htaccess
> would do fine.

That's not a safe assumption.  To begin with the server config file has
to permit the .htaccess file to override previous access directives. 
Taking it one step further, there's no guarantee the server is even
configured to use ".htaccess" as the filename.

I would recommend asking your provider for config assistance.

At the very least you I would suggest you start by reading the docs at 
  http://httpd.apache.org/docs/

  Specifically:
  Auth/Access
  http://httpd.apache.org/docs/howto/auth.html
  FAQ: htaccess
  http://httpd.apache.org/docs/misc/FAQ.html#htaccess-work

> 
> Please remind me of the syntax for the .htaccess to at least prevent
> robots from caching the files, or, better yet (I think), permitting ssl
> connectivity.  Ideally, if I could be shown both (.htaccess and ssl), if
> one doesn't work right, I'll try the other.

.htaccess isn't the place to control crawler access or enable SSL. 
Again, I would refer you to the docs at the above link.

If all you want to do is tell crawlers to stay out, check out the
robots.txt specification at: http://www.robotstxt.org/wc/exclusion.html

Keep in mind this only works for "well behaved" crawlers and bots.

Depending on your needs you may also be able to do what you want by
setting up a blind path.  i.e. a path that does not have a link pointing
to it.

e.g. set up http://foo.bar.com/lesspublic/....  As long as none of the
files in http://foo.bar.com/ point into "lesspublic" a crawler will not
be able to drill down to it.  Anyone who knows about the "lesspublic"
directory will still be able to access your content.

  - Paul Beltrani

> 
> Also, if someone could show me how to test the implementation, that would
> be most helpful, too.
> 
> Thanks.
> 
> Scott
> 
> _______________________________________________
> bblisa mailing list
> bblisa at bblisa.org
> http://www.bblisa.org/mailman/listinfo/bblisa
>