Interactive Memcached Debugging with Rails

Posted on June 20, 2009

The memcached support in Rails is great, but it’s a bit difficult to see what the heck is going on in a production environment. The first problem is, by default Rails doesn’t keep a list of all memcached keys that are available on the system. So, you have to use a plugin like memcache_store_with_delete_matched to have memcache itself store a list of all available keys.

You can install this plugin with the typical script/plugin command.

script/plugin install git://github.com/lacomartincik/memcache-store-with-delete_matched.git

Then, you configure memcached in your environment.rb like this.

mem_cache_options = {
    :c_threshold => 10000,
    :compression => true,
    :debug => false,
    :timeout => false,
    :namespace => 'app',
    :readonly => false,
    :urlencode => false
  }
  config.action_controller.cache_store = :mem_cache_store_with_delete_matched, ['127.0.0.1:11211'], mem_cache_options

The other benefit to using memcache_store_with_delete_matched is that regular expressions within expire_fragment works.

expire_fragment(%r!/index_user_.*!)

So, down to debugging. If you want to see all keys in a script/console, you can do the following.

YAML.load(ActionController::Base.cache_store.fetch("memcached_store_key_list"))

You can select certain keys from the memcached_store_keylist.

YAML.load(ActionController::Base.cache_store.fetch("memcached_store_key_list")) .detect{ |k| k.match(/index/) }

Once you have the list of keys, you can plug them into a fetch command.

ActionController::Base.cache_store.fetch("app/blog/index")

You can also manually expire keys from a console.

c = ApplicationController.new
c.expire_fragment(/index/)

Using YUICompressor with Capistrano and Rails 2.3+ on Combined Javascript and CSS

Posted on June 18, 2009

YUICompressor is a standalone Javascript and CSS minifier from the YUI folks. It’s fairly awesome in that it does deep analysis of Javascript using Rhino (a standalone Javascript interpreter). Most minifiers take the low road and merely remove spaces and newlines, and, if you’re lucky maybe shorten the variable names. YUICompressor one-ups them all because it actually parses all your JS and aggressively optimizes it for download speed.

CSS and Javascript Conflation

The problem is most of the guides out there for integrating YUICompressor with Capistrano and Rails are woefully out-of-date. The main issue is that Rails now has built-in support for combining all JS files into a single file. So, there’s no need to manually do this in Capistrano these days.

You can make Rails combine automatically combine your Javascript by adding a :cache => "cache/all" to your javascript_include_tag and stylesheet_link_tag in your layout.

<%= stylesheet_link_tag 'one', 'two', :cache => "cache/all" %>
<%= javascript_include_tag 'jquery', 'ninja',  :cache => "cache/all" %>

Priming the JS and CSS Caches

The Capistrano stuff is a bit tricky because Rails doesn’t generate the Javascript and CSS until after the app has been visited for the first time. So, you have to visit each of your web servers after restart during your deploy.

task :after_restart, :roles => :web do
  desc "Visiting each web server"
  sleep(5)  # Wait for passenger to fully spin-up
  run "/usr/bin/wget -O- http://127.0.0.1 >/dev/null 2>&1"
end

Invoking YUICompressor

Now, we can invoke YUICompressor after after_restart. The previous after_restart task automatically gets invoked after the web servers are restarted, however, the compression needs to happen after that. So, you have to manually chain a minify just after after_restart.

after "deploy:after_restart", "deploy:minify"

The actual task is quite simple for running the compressor itself. It’s just a normal Capistrano task.

task :minify, :roles => :web do
  desc "Minify JS and CSS Using YUICompressor"
  javascript = "#{current_path}/public/javascripts/cache/all.js"
  stylesheet = "#{current_path}/public/stylesheets/cache/all.css"
  compressor = "java -jar /usr/lib/java/yuicompressor-2.4.2.jar"
  run "#{compressor} --type js #{javascript} -o #{javascript}"
  run "#{compressor} --type css #{stylesheet} -o #{stylesheet}"
end

Panasonic TZ5 HDRs

Posted on June 15, 2009

For being such a small camera, the Panasonic TZ5 does a pretty decent of job shooting bracketed images. Bracketing is where you shoot three differently exposed shots and then use software like Photoshop CS4 or Photomatix Pro to blend them together to produce a high-dynamic range (HDR) photograph. Over the past year, I’ve been experimenting with bracketing, HDR, and various software to produce some fairly stunning results. Out of all those, these are my greatest hits.

Austin Creek Fire Road

This shot was taken near Guerneville, CA at the Austin Creek State Recreational Area which is adjacent to Armstrong Redwoods State Natural Reserve. Taking pictures under a forest canopy is tricky. And, to be honest, most of the shots I’ve attempted in these conditions normally don’t amount to anything. Doing HDR and applying some “highlight” adjustments create a stunning result about 10% of the time. I was lucky here because this turned out the best out of anything I’ve shot in the ubiquitous redwood parks surrounding the Bay Area.

San Francisco Park Presidio

The Presidio is another area where it’s difficult to take decent photographs due to the sun shining down through gaps in the canopy. It’s similar to the redwood parks in that images tend to be overexposed in parts and underexposed in others due to beams of sunshine sparsely illuminating the groves of trees. This image turned out quite well, but there’s some rippling at the bottom you can see at high-resolutions due to the fact I was sans tripod on this walk.

Foggy Golden Gate Bridge

The following HDR I took from the Marin Headlands. It was a bit of a chore finding a place to sit the camera on the mini-tripod. My normal procedure is to set a two second timer and leave the camera resting on a fence post before shooting an HDR up there. Small movements in the exposure can wreak havoc with Photomatix Pro. On this one, I feel like I turned the saturation up a bit too high, but people seem to like it.

Here is a link to all of my other HDR images.