Posted by Luke Ludwig
Sun, 17 Feb 2008 04:57:00 GMT
It is very easy to consume a ton of memory when using ActiveRecord's find(:all) method. When transitioning the
Team Sport Tech rails app from using file column to attachment_fu I wrote a migration to convert all of the photos we had to the new database format and the new location on disk. Without thinking I wrote code like this:
Photo.find(:all).each do |photo|
# conversion commands
.
.
end I didn't notice any issues executing this on my laptop which has 2 GB of RAM, but when I went to run this migration on our staging server over at
Engine Yard I had problems. Our staging server only has 640 MB of RAM. Our database has over 100,000 photos. That is an array with 100,000 ActiveRecord objects in memory all at once. As the migration executed on the staging server I could see that the rake task was using all available RAM and it was paging like crazy, utilizing 600 MB of virtual memory from disk. The cpu fluctuated between 0 and 1 percent of utilization due to the paging. If adequate memory was available this migration would take around 10 minutes and the cpu would be working like crazy. I did a few other things for awhile and came back to this migration 2 hours later. Still working and it probably had a long ways yet to go.
Read more...
Posted in Programming | Tags ActiveRecord, apply_to_all, attachment_fu, rails, ruby, will_paginate | no comments | no trackbacks
Posted by Luke Ludwig
Sun, 17 Feb 2008 00:35:00 GMT
Most Rails applications have to deal with resizing uploaded images for the creation of thumbnails. The main choices include RMagick, MiniMagick, or ImageScience, all of which come packaged as gems. Alternatively you can write your own which really isn't that difficult. So which one should you use? First I would recommend not writing your own, because it is really nice to take advantage of one of the very fine attachment plugins that are available. File column is the old standby rails attachment plugin, but it uses RMagick. Attachment_Fu is more flexible since you have the choice of using RMagick, MiniMagick, or ImageScience and can switch between them easily. At TeamSportTech where I work I recently transitioned our relatively large rails application from using file column and RMagick to using attachment_fu and ImageScience. Originally I was planning on using MiniMagick instead of ImageScience, but it turns out that MiniMagick is quite slow. The following timing results are on my Mac Book when running in development mode, and include uploading a single 3.4 MB jpeg which is resized down to 4 different sizes. MiniMagick consistently took 18 seconds to accomplish this, RMagick 7 seconds, and ImageScience 6 seconds. This doesn't accurately represent a production environment, but I do believe it is a fair comparison. Note that the time to upload is not a factor since this was done entirely on my local computer.
So why is MiniMagick with attachment_fu so slow? And why not use RMagick? RMagick and MiniMagick use the well known ImageMagick C libraries. RMagick works by providing API ruby bindings to the ImageMagick libraries, which means that RMagick operates within the ruby process. It is well documented that RMagick consumes a lot of memory and has memory leaks as well. See Craig Ambrose's article and a Mephisto article. It appears like RMagick2 provides better memory management.
Read more...
Posted in Programming | Tags attachment_fu, file column, FreeImage, ImageMagick, ImageScience, MiniMagick, rails, RMagick, ruby | no comments | no trackbacks
Posted by Luke Ludwig
Thu, 06 Dec 2007 01:35:00 GMT
Sending email from a web application, especially blast emails to a lot of people, can take a lot of time. Generally you don't want the user to wait until all the emails have been handed off to the smtp server. You also probably don't want to tie up an entire mongrel with sending mail. The ar_mailer gem solves this problem in excellent fashion, by saving pending emails to the database and having a separate ruby daemon process periodically check the database and send emails. I recently set up one of our rails apps at work to use ar_mailer. Configuring it to use ar_mailer was incredibly easy, but it was tricky to get the ar_sendmail ruby daemon process to run under monit. On our production servers which we have hosted at Engine Yard, we want every process that our application depends on to be monitored by monit.
The primary feature that ar_sendmail lacks to play nice with monit is the ability to leave a pid file after it starts up and to remove it when the process exits. This has already been pointed out on rubyforge as a feature request. Here is what I did to get ar_sendmail working under monit: (ar_mailer 1.3.1)
Read more...
Tags ar_mailer, ar_sendmail, monit, rails | 5 comments | no trackbacks
Posted by Luke Ludwig
Mon, 26 Nov 2007 23:59:00 GMT
For a rails app at work we store user access privileges in the session. This is done as an optimization to avoid an extra sql query that would need to be done for every page view to determine if the user has edit privileges. Security-minded people may see this as a security hole. For this application I don't see this as a big deal. Its not like our rails app is controlling the launch of nuclear missiles.
The problem is that when an admin goes to modify the user access privileges for someone, the changes won't take affect until the user next logs in since the user access privileges are stored in the session. So if the user is already logged in this is a problem. They will have to log out and log back in for the changes to take affect. The solution to this problem is to modify the session of the user who's access privileges were modified.
To do this we need to add a user_id column to the sessions table. This can be done like any other migration. The tricky part is accessing the user_id column. We will want to set the user_id of the session when someone logs in. This of course will not work:
session.user_id = @user.id
Read more...
Posted in Programming | Tags ActiveRecordStore, rails, sessions, SqlSessionStore | 3 comments | no trackbacks