category programming

Switching from mongrel to mod_rails.

Posted by Luke Ludwig Sun, 13 Apr 2008 14:40:00 GMT

A mod_rails for Apache, also called Phusion Passenger, has been released. I switched this blogs deployment from using mongrel to using mod_rails in less than 20 minutes without any problems. The installation procedure is dead simple. I had to look up one thing in the user's guide, how to serve the rails app off a sub path of the domain (lukeludwig.com/blog), and I found it quickly. Overall I am extremely impressed and am curious how Passenger will do with higher traffic production sites.

Previously I was running this blog on a single mongrel instance, which is all that is needed due to the low traffic on this site. But lets just say that I wrote an article with great content and it got digged, causing traffic to spike like crazy for a few days. The single mongrel would quickly be overwhelmed and my site would be for the most part, unusuable, until I noticed and was able to start up a whole mongrel cluster of say a dozen mongrels. My server has 1 GB of RAM, most of which goes unused. Passenger is able to handle the number of rails instances in a much nicer manner. From a configuration perspective, I set the RailsMaxPoolSize to 12 and I'm done. Passenger will run as many rails instances as it needs depending on load, up to the max of 12. When traffic slows down, it kills the idle rails instances to conserve memory. Great stuff.

Posted in Programming | 1 comment | no trackbacks

Use pagination to save memory when iterating over a lot of ActiveRecord objects.

Posted by Luke Ludwig Sun, 17 Feb 2008 04:57:00 GMT

It is very easy to consume a ton of memory when using ActiveRecord's find(:all) method. When transitioning the Team Sport Tech rails app from using file column to attachment_fu I wrote a migration to convert all of the photos we had to the new database format and the new location on disk. Without thinking I wrote code like this:

    Photo.find(:all).each do |photo|
      # conversion commands
      .
      .
    end

I didn't notice any issues executing this on my laptop which has 2 GB of RAM, but when I went to run this migration on our staging server over at Engine Yard I had problems. Our staging server only has 640 MB of RAM. Our database has over 100,000 photos. That is an array with 100,000 ActiveRecord objects in memory all at once. As the migration executed on the staging server I could see that the rake task was using all available RAM and it was paging like crazy, utilizing 600 MB of virtual memory from disk. The cpu fluctuated between 0 and 1 percent of utilization due to the paging. If adequate memory was available this migration would take around 10 minutes and the cpu would be working like crazy. I did a few other things for awhile and came back to this migration 2 hours later. Still working and it probably had a long ways yet to go. Read more...

Posted in Programming | Tags ActiveRecord, apply_to_all, attachment_fu, rails, ruby, will_paginate | no comments | no trackbacks

RMagick has memory problems. MiniMagick with Attachment_Fu is slowwww. Go ImageScience.

Posted by Luke Ludwig Sun, 17 Feb 2008 00:35:00 GMT

Most Rails applications have to deal with resizing uploaded images for the creation of thumbnails. The main choices include RMagick, MiniMagick, or ImageScience, all of which come packaged as gems. Alternatively you can write your own which really isn't that difficult. So which one should you use? First I would recommend not writing your own, because it is really nice to take advantage of one of the very fine attachment plugins that are available. File column is the old standby rails attachment plugin, but it uses RMagick. Attachment_Fu is more flexible since you have the choice of using RMagick, MiniMagick, or ImageScience and can switch between them easily. At TeamSportTech where I work I recently transitioned our relatively large rails application from using file column and RMagick to using attachment_fu and ImageScience. Originally I was planning on using MiniMagick instead of ImageScience, but it turns out that MiniMagick is quite slow. The following timing results are on my Mac Book when running in development mode, and include uploading a single 3.4 MB jpeg which is resized down to 4 different sizes. MiniMagick consistently took 18 seconds to accomplish this, RMagick 7 seconds, and ImageScience 6 seconds. This doesn't accurately represent a production environment, but I do believe it is a fair comparison. Note that the time to upload is not a factor since this was done entirely on my local computer.

So why is MiniMagick with attachment_fu so slow? And why not use RMagick? RMagick and MiniMagick use the well known ImageMagick C libraries. RMagick works by providing API ruby bindings to the ImageMagick libraries, which means that RMagick operates within the ruby process. It is well documented that RMagick consumes a lot of memory and has memory leaks as well. See Craig Ambrose's article and a Mephisto article. It appears like RMagick2 provides better memory management.

How to add a new column to Rails' sessions

Posted by Luke Ludwig Mon, 26 Nov 2007 23:59:00 GMT

For a rails app at work we store user access privileges in the session. This is done as an optimization to avoid an extra sql query that would need to be done for every page view to determine if the user has edit privileges. Security-minded people may see this as a security hole. For this application I don't see this as a big deal. Its not like our rails app is controlling the launch of nuclear missiles.

The problem is that when an admin goes to modify the user access privileges for someone, the changes won't take affect until the user next logs in since the user access privileges are stored in the session. So if the user is already logged in this is a problem. They will have to log out and log back in for the changes to take affect. The solution to this problem is to modify the session of the user who's access privileges were modified.

To do this we need to add a user_id column to the sessions table. This can be done like any other migration. The tricky part is accessing the user_id column. We will want to set the user_id of the session when someone logs in. This of course will not work:

session.user_id = @user.id

A Virus the World Needs : How to Topple the Reign of Internet Explorer

Posted by Luke Ludwig Sat, 20 Oct 2007 22:39:00 GMT

For my first project at my new job, a Ruby on Rails position with Team Sport Technologies, I implemented a table builder, which allows a user to create simple tables using their browser without writing HTML or any other special markup. My JavaScript was a bit rusty, so it took a few days before I had the table builder complete. I was working with a brand new Mac Pro without Parallels installed, and so I had yet to try the table builder in Internet Explorer, but it worked perfectly in Firefox, Safari, and Opera. I soon found out that it failed miserably in Internet Explorer. I was assuming it would be a quick fix, but it turned out that Microsoft's implementation of the DOM Event model is so messed up that I had to redo the entire implementation using a different approach.

Internet Explorer is a source of constant frustration and wasted hours of web developers across the world. All other web browsers follow standards set by the W3C World Wide Web Consortium describing how web browsers should interpret Cascading Style Sheets, JavaScript, and the Document Object Model. Books on these subjects dedicate dozens of pages to detailing the crazy hacks that programmers need to use to get their web sites to operate in Internet Explorer. Microsoft breaks many of these standards intentionally, and can get away with it simply due to having a majority of the browser market.

Erlang - The Next Great Programming Language?

Posted by Luke Ludwig Thu, 04 Oct 2007 13:13:00 GMT

Erlang - The Next Great Programming Language? I don't think so. I probably shouldn't be doing this. At least not yet. I just don't know enough about Erlang to make any predictions. I haven't written any Erlang code myself. This will change shortly, as the book is on its way. I guess I am impatient. And curious to see if my opinion will change.

All signs seem to point to Erlang being the next great programming language. It has momentum, it has hype, it even has the Pragmatic Programmers on its side. And like Ruby, it has a killer app -- parallel processing. Ruby's killer app is Rails of course, which greatly eases the burden in developing web applications. Ruby is the latest great programming language. For the past 3-4 years Ruby (and Rails) is the language that everyone has been talking about, the language that all the talented programmers want to learn in their spare time. Before Ruby, Python may have stood for a short time on the latest great programming language podium. Python never quite had the killer app that Ruby does with Rails, but it still shocked the world (well, the world of nerds anyway) with its pseudocode look and its use of whitespace. Before Python there was Java, and before Java C++, and before C++ there was C. I wasn't programming at the time, but yes people I believe if you could travel back in time there was a point where C was the latest cool programming language that all the cool kids (aka major nerds) had to try out.

And now Erlang? Certainly it seems like all the cool kids are sneaking off to become Erlang experts during their lunch break. Which is the beginning of how the next great programming language comes to be. Due to its killer app, parallel processing, Erlang is impossible to ignore. Multi-core computers are now a reality, with more cores coming out every year. Intel could have 32 cores by 2010. The tides have turned and the software world now lags behind the hardware world. In the past we always had software programs that would consume the available processor time or eat up all the RAM. But now, with 32 cores just around the corner, very few software programs will be able to take advantage of this. Unless Erlang becomes the next great programming language, and quickly.

Posted in Programming | Tags Erlang | no comments

There is more to life than code.

Posted by Luke Ludwig Wed, 03 Oct 2007 03:34:00 GMT

Many skilled programmers are good, in part, because it is all they do. At work they code and they can't wait to get home so they can code on their own project. The world of programming is a large turbulent pool of knowledge, ideas, and skillsets too vast and changing for any one human mind to conquer. For many people this world of code is their primary passion in life. Programmers get sucked into an hour-gobbling vortex that leaves little time for other activities. Tired of sitting at your desk all day? Followed by sitting in your car as you commute home, and further by sitting in front of the tv or computer at home? Sit, sit sit, its what Americans do, especially lazy but passionate-about-code programmers. I think it is important for the health of the human psyche to have at least one hobby or activity, unrelated to work, that gets you excited, and why not make it a physical activity since the majority of your life is spent sitting on your bottom?

Physical activity? Most people are turned off just by the thought of it. Physical activity is usually set aside as one of those healthy things you should be doing, but never have time for. Like flossing your teeth. Occasionally inspiration hits and you'll go for a run every other day or hit the weight room. But if you are like most people, this usually lasts for only a week or two before you are back on the couch. You wouldn't want to miss an episode of Heroes. The problem is that very few people actually enjoy running or lifting weights. What you need is a physical hobby which excites you as much as the latest cool programming language. If you are passionate about excelling at a physical hobby, then your own self-motivation will drive you to stay in great shape, just as it drives you to stay on top of the latest programming trends.

Inspiration While Kayaking

Posted by Luke Ludwig Fri, 04 May 2007 00:23:00 GMT

One Thursday afternoon in early May, I decided to leave work early at 3:30 and go kayaking. Adios manager Chris. After a boring hour commute home to lovely Woodbury, Minnesota, I tossed my kayak on top of my truck, drove to nearby St. Croix Bluffs Regional Park, launched, and was on the water by 5:00. My kayak is a yellow 17 foot long Current Designs Caribou and is designed for "seas" as it is a sea kayak. I paddled across the wide St. Croix river to the Wisconsin side and headed downriver along the shore. Out for exercise, I was planning to paddle about 8 miles in a moderately strenuous manner. The planned 4-mile turn around point is where the St. Croix joins with the Mississippi.

I soon got in a rhythm and began thinking about, what else -> backpacking. If you were to just meet me and ask what I do I would say "I backpack, paddle, and write code" [1]. Certainly you were expecting an answer more like, "I work as a Simulation Engineer at BAE Systems in Fridley. We make the biggest guns in the world, aka howitzers, for the army." Backpacking, paddling kayaks and canoes, and writing code are my three main... passions or hobbies? If I just call them passions then I have to make the standard disclaimer about my wife Becca being my first passion. How about passion-hobbies.

Mad Marmot

A blog about programming, ruby, rails, and my crazy outdoor pursuits