DeviceGuru

Since its launch last December, DeviceGuru.com’s stories have made it onto Slashdot about once a month. Unfortunately, the site’s “budget” web-hosting service balks at delivering the resulting traffic spikes. What to do?

Of course, there are many ways to optimize web pages so their web server load is minimized. But even if you have the knowhow to tweak your site’s page format, database, and web server parameters, getting a story on Slashdot or Digg will probably generate tens of thousands of visits to your site in a single day, resulting in data transfer rates that exceed the peak levels available from most cheap web-hosting accounts.

Fortunately, there’s a relatively easy fix.

CoralCDN

The solution hinges on a free service known as the Coral Content Distribution Network (CoralCDN).

CoralCDN is described by its originators as “a decentralized, self-organizing, peer-to-peer web-content distribution network.” Basically, the network comprises a large number of Internet-connected systems that cache web pages and serve them to client systems’ web browsers, rather than requiring the pages to be delivered directly from the original source. (For a detailed technical explanation of how this happens, visit this page on CoralCDN’s website.)

So how do you get CoralCDN to serve your pages instead of having them originate from your website?

Suppose your website is mywebsite.com, and you’re about to publish an article (”hot_news.html”) that you expect to produce high levels of traffic. Let’s assume the normal address of that article would be “http://mywebsite.com/news/hot_news.html.”

If you publicize that address, your website will bear the full brunt of all the traffic the article generates. To get CoralCDN to handle the load for you, all you need to do is make a small change to the story’s URL. To do this, insert “nyud.net:8080″ into the story’s address, right after the “.com” part of the address, like this:

http://mywebsite.com.nyud.net:8080/news/hot_news.html

That’s all there is to it. To see this in action, click the following “Coralized” version of DeviceGuru’s address:

DeviceGuru.com (via CoralCDN)

To reduce the load on a story using CoralCDN, you can simply place the story’s Coralized address on your home page or news page instead of the story’s native address, and CoralCDN will do the rest. Similarly, when you submit your hot story to Slashdot, Digg, or another high-traffic social networking site, submit the Coralized URL instead of the native one.

Oh, and one more point: if your hot story has images or other pages on your website referenced in it, you might want to modify their URLs so that they, too, won’t generate significant traffic loads on your site.

Automating the process — the hard way

As described above, a comprehensive site wide CoralCDN-based traffic reduction fix would be a time consuming, manual task.

It’s also possible, however, to automate the process so that whenever page requests on your site are generated by referrals from sites like Digg or Slashdot the URL is Coralized on the fly. I’m aware of at least two ways this can be done, but there are probably others.

One approach is to utilize various Apache code modules in conjunction with statements added to Apache’s configuration files that instruct the web server to redirect page requests to Coralized addresses. Look here for hints on how to do this using Apache’s .conf files, and here for suggestions on doing it with .htaccess files. On a cheap web-host, you’re unlikely to be able to modify the .conf files, so the second approach (using .htaccess) would be the one to use.

Automating the process — the easy way

Unfortunately, I’m not knowledgeable enough regarding Apache and its configuration to implement the method described above. Thankfully, however, at least one popular blogging platform — WordPress, DeviceGuru.com is based on — has an easily installed plugin that accomplishes exactly what’s needed.

Digg Defender, by Elliot Back, performs on-the-fly Coralization of a WordPress site’s URLs for page requests referred from digg.com, slashdot.org (and .com), fark.com, somethingawful.com, kuro5hin.org, engadget.com, boingboing.net, and del.icio.us.

This handy plugin has been around for several years and, in DeviceGuru.com’s experience, works extremely well. The plugin’s author notes one caveat, though, that “Coral Cache is hit/miss and can be slow sometimes to load.” While I’ve also noticed some CoralCDN unreliability along these lines, I’d much prefer to take that chance rather than risk having my site be unable to deliver a hot story to eager readers due to my budget web-host’s peak bandwidth limitation being exceeded!

One handy feature of Digg Defender is that it can be easily enabled/disabled using the WordPress Dashboard’s “Plugins” tab. I make a habit of only enabling Digg Defender when I expect to need it. Additionally, before enabling Digg Defender I make sure CoralCDN is up and running by using this link.

Since installing Digg Defender, DeviceGuru.com has had its stories featured on Slashdot several times without noticeable degradation, with the exception of one instance when CoralCDN was briefly down. I have to say a big “thank you” to both the folks behind CoralCDN, and to Digg Defender author Elliot Back.