ianneubert.com

Here is an rsync command that will synchronize music stored in a Mac to a PC. It will not sync Movies or TV Shows. Just remove the –excludes if you want to sync those things.

rsync -av --force --delete --stats --size-only -e ssh --exclude="ethumbs_vista.db" --exclude="Thumbs.db" --exclude=".DS_Store" --exclude="iTunes\ Library.xml" --exclude="Mobile\ Applications/*" --exclude="Movies/*" --exclude="TV\ Shows/*" ~/Music/iTunes/* /Volumes/itunes

Thanks to MacRumors for the base of this command.

Ever wanted to take the last 1000 lines of a log file from your remote server and then copy that to your local machine to use in a nicer text editor?

Now you can! This single command will grab the last X lines from a remote file, remove any ANSI color codes, and then compress the output right on to your local computer.

Just replace the green text with your information and you’re all set.

ssh username@my.server.com "tail -n 3000 /var/log/mylogfile.log | sed -r 's/\x1B\[([0-9]{1,3}((;[0-9]{1,3})*)?)?[m|K]//g' | bzip2" > /my/local/file.log.bz2

The log will be trimmed, cleaned, and compressed all on the remote server before transport.

I just now got around to installing RVM on my Mac. I wish I had done it a lot sooner!

RVM let’s you install multiple Ruby versions side by side. You can easily install a new ruby version just by typing:

rvm install ruby-1.8.7-p160

or

rvm install 1.9.2-preview1

And you can quickly switch between versions by typing this:

rvm ruby-1.8.7-p249

Note that you will need to install gems independantly in each of your ruby environments. But they make it easy with gemsets. Also note, that if you want to preserve your custom ENV settings on your Mac, you should place the bash source line into your .profile file and not to .bashrc as it notes.

Have fun!

I previously published a Ruby script that essentially duplicates an Amazon S3 bucket. I use it to copy my production bucket into a testing bucket that I can then use in my staging environment, without fear of killing off something important.

Anyway, this script used to take around 18 minutes to complete a full copy of a bucket with about 2,000 keys. What a pain when I had to run this script multiple times for some testing!

So I finally got around to rewriting it today, and wow, what an improvement!

I upgraded the delete and copy operations to run multithreaded, and boy does it fly.

On my main bucket (with about 2,000 keys) it now only takes about 1.5 minutes to complete! That’s 12x faster than it used to be!

So if you are looking for a multithreaded Amazon S3 bucket copier or duplicator, check it out!

Continuing my new trend of publishing code that I use, I’ve released a new fork of RightScale’s right_aws ruby gem. This gem makes accessing Amazon’s S3, EC2, and other web services a piece of cake through ruby.

One problem with (at least) the latest release of right_aws is that it tries to connect to the Amazon API at the bucket_name.s3.amazonaws.com. This causes the connections to the API to thrash when you are working with more than one bucket. I’ve taken out the code that makes that connection so that it will now only connect to s3.amazonaws.com/bucket_name by default.

You can find the code on GitHub. And see extra documentaion here.

I just published some more code on github today. This new repo contains some ruby code that I use to manipulate my Amazon S3 buckets.

Right now there are just two simple scripts, one deletes a whole bucket, the other copies everything from one bucket to another, with an optional prefix to limit what is copied.

I’ve been using these scripts for a few months now, they are very stable and do some basic error handling.

Here is the README file.