Changes between Version 1 and Version 2 of WebserverSetup


Ignore:
Timestamp:
08/12/11 14:12:22 (13 years ago)
Author:
joe
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • WebserverSetup

    v1 v2  
    1311313. If it didn't have a DNS name, you can try using the `whois` command to see who institution the IP range is registered to. 
    132132 
    133  
     133=== Blocking by Browser / User-Agent === 
     134 
     135You can also reject by the client's browser identification string, although, they can always change or hide it, so this isn't always good solution.  It can be a hint to search crawlers or `wget` users that their activity isn't appreciated, though. 
     136 
     137As we have hundreds of thousands of requests per day for the same file, all coming from thousands of IP addresses, by a single strange browser (that I've tried asking companies with similarly named software if their stuff acts as a browser, and if so, to stop it), we use the following to block access: 
     138 
     139{{{ 
     140<Location "/images/latest_eit_304.gif"> 
     141    BrowserMatch ^CompanionLink badClient 
     142    Order allow,deny 
     143    Allow from all 
     144    Deny from env=badClient 
     145</Location> 
     146}}} 
     147 
     148== Restricting Access to Local Users == 
     149 
     150If you have data that's under embargo, and you want to make it available only to local users, you can limit access to a directory such as `/embargoed` to [http://httpd.apache.org/docs/2.0/mod/mod_access.html#allow specific IP addresses or to group of IP addresses]: 
     151 
     152{{{ 
     153<Location "/embargoed/"> 
     154    Order deny,allow 
     155    Deny from all 
     156    Allow from 10.1.1.1/24 
     157    Allow from 10.11.12.167 
     158</Location> 
     159}}} 
     160 
     161If you want to allow the data to be accessed by local users, by also from outside with the proper username and password, you can do: 
     162 
     163{{{ 
     164<Location "/embargoed/"> 
     165    Order deny,allow 
     166    Deny from all 
     167    Allow from 10.1.1.1/24 
     168    Allow from 10.11.12.167 
     169 
     170    AuthType Digest 
     171    AuthName "Embargoed Data" 
     172    AuthDigestDomain /embargoed/ 
     173    AuthDigestFile /path/to/htdigest/file 
     174    Require valid-user 
     175 
     176    Satisfy any 
     177</Location> 
     178}}} 
     179 
     180Note that in Apache 2.2, `AuthDigestFile` should be changed to `AuthUserFile`.  You can use `AuthType Basic` instead, but then the passwords are sent in the clear, and you'll have to [http://httpd.apache.org/docs/2.0/howto/auth.html change some of the lines]. 
     181 
     182 
     183 
     184 
     185== Debugging Slow Web Servers == 
     186 
     187Entries into the webserver's access logs only occur once the client disconnects.  This means that if you have lots of connections sitting open, you can't use the access logs to see what's going on.  Apache does have a way to get some information about what's going on, however.  Look for a section in your server config mentioning `ExtendedStatus` or `server-status`, and change it to read something like: 
     188 
     189{{{ 
     190<IfModule mod_status.c> 
     191    <Location "/server-status"> 
     192        SetHandler server-status 
     193        Order deny,allow 
     194        Deny from all 
     195        Allow from 127.0.0.1 
     196        Allow from your-ip-address 
     197    </Location> 
     198    ExtendedStatus On 
     199</IfModule> 
     200}}} 
     201 
     202Obviously, set `your-ip-address` to an appropriate value, and you can set more than one `Allow` line to allow connections from more than one machine.  You can then (after restarting the webserver) request the page `http://servername/server-status`, which down at the bottom will give a report that includes what requests are being processed, what IP asked for it, and how long it's been processing, so you can try to identify what might be having problems, or what connecting IP address might be doing strange things.  You have to do this in advance of the request; as it requires a web server restart to turn on this feature, you can't just turn it on when you have a problem connection (unless it's a flood of problem connections that you're actively monitoring). 
     203 
     204  
     205