Red Letter Day

This blog (and the other blogs and domains on my master account) are not very popular (in spite of the general awesomeness which pervades every pixel). Our monthly bandwidth is a couple gigabytes at best, which is why I was very surprised yesterday morning when I got an automated letter from my hosting provider telling me I was on a path to blow through my monthly allotment of 150 gigs of bandwidth and be liable for a big overage charge!

The culprit was one of those slimy, scammy “you won’t believe what this video showed the babysitter did when the parents were away” sites. They were direct-linking to the original of a tiny 25 k png image that Dave uses for his site (and he has copyright of the image, adding insult to injury!) Downloaded, oh, a few million times, that adds up.

There’s a few ways to deal with this. One obvious and fun way would be to simply replace the original image with one that perhaps contained a double bird and insulted the thief’s mother, but, as satisfying as that would have been, it still would take my bandwidth. Another option would be to simply rename the image, breaking their IMG SRC tag, but while this would stop this specific thievery, it wouldn’t stop them (or anyone else) from figuring out the new image name and using it instead.

I needed a way to stop all external referrer image linking to my account, but still allow images to be referred when the page was locally hosted (i.e. part of my blog).
In other words, this will not allow someone to use your image as part of their site, directly from your server (normal hyperlinks to your site work the same as always).

After a fast and intense Google-powered brain-bang, I had found the answer!

The way to do this is via an .htaccess file that utilizes a built-in feature of the Apache web server called mod_rewrite.

You create a file called “.htaccess” at the very top level of the web site you want to protect (or append the the existing one if it is already there), and put the following text in it:


RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://([^.]+\.)*yourdomain\.com/ [NC]
RewriteCond %{HTTP_REFERER} !^http://yourblogspotblog\.blogspot\.com/ [NC]
RewriteRule \.(jpg|gif|png|bmp|mp4|avi|mp3)$ – [F]


Replace “yourdomain” with the actual name of your domain (and obviously replace ‘com’ with ‘org’ or whatever if it is a .org site. The * is a wildcard, covering prefixes like “www” or whatever, as well as the naked, raw URL.

You can have as many lines as you have domains you wish to allow linking from. In other words, this is a whitelist of allowed domains, generally ones you own or post to. I’ve included a blogspot blog here too, for example, if you have a blogspot blog from which you link images you host on your main domain.

Make sure to keep the backslashes, carets and other goop intact, they are used as part of the regular expression.

The last line lists the file extensions that you are not permitting to be externally linked. In my case, I want to prevent links to common graphic, music and movie formats.

Save your .htaccess file and you should be good to go – it should take effect immediately.

Now, you will want to test your changes.

You will need access to a “non-allowed” domain. If you have a friend with a web site, ask to use it, or you can always use wordpress.com or something. To test, just create some HTML code that directly links to a file on your protected site, a normal IMG SRC or whatnot.

Save it, and clear your local browser cache – this step is very important, because if the image is in the cache somewhere, it will still be displayed even if the .htaccess file is working great. Then load the test page. You should see broken image indicators for the images.

If not, make sure again to clear your browser cache (or try on another machine), and check the .htaccess file to make sure the code is correct and it has proper permissions (644 – world readable, but only writeable by the owner).

Lastly, don’t forget to verify that images do show up properly from within your own site. If you made a typo in your domain name when editing the .htaccess file, this would be the result, so double-check with all the “whitelisted” domains.

So yesterday I needed an easy, quick way to track and log the memory use of an application remotely, in (near) real-time, as I performed various actions.

I could SSH in and run something like top, but that doesn’t get me the logging. There are several graphic “activity monitor” like utilities including, well, Activity Monitor and Big Top, but those don’t work remotely and I didn’t want to graphically control the remote machine since that performance was part of what I was measuring.

Apple’ Instruments would have been really great, but it doesn’t work remotely, at least in any way that was quick or obvious.

After some brief searching, I found a great too, written in Python, called Syrupy, written by a guy named Jeet Sukumaran. It was tiny, simple, lightweight and does only one thing – records the state of a process over time. You tell it which process(es) you want to monitor, and how often to sample them, and it will log the results to a file or the console.

If you choose the latter, you can even make quick and dirty test notes “in line” which makes it really easy to go back later and remember what you did/see the effect of what you did on memory or processor use. You can also control the output format with a few arguments when you invoke Syrupy, which can be useful if you have some graphing app that is picky about formatting.

You run it on the remote machine via an SSH session (obviously you can use it locally too).

Default output looks like this:

SYRUPY: Writing raw process resource usage logs to 'syrupy_20100726160016.ps.raw'
SYRUPY: sampling process 169
PID DATE TIME ELAPSED CPU MEM RSS VSIZE
169 2010-07-26 16:00:16 02:08 0.0 0.1 4932 409880
169 2010-07-26 16:00:26 02:18 0.0 0.1 4924 409360


Anyway, Syrupy can be found here if you need it.