Ben Reichelt’s Weblog

This post from Jason Bock reminded me that I never followed up on my Amazon S3 backup plan.

In my last post, I mentioned that I was ditching JungleDisk to use S3Drive because it didn’t cache the files locally, and it mirrored my file structure with no encryption which makes it convenient to use.

After trying to backup my files with S3 Drive, I’ve now made the switch back to JungleDisk, primarily because the file transfer speeds with S3Drive were horrendous (~2k/sec).  At those speeds doing a backup of all our pictures would have taken days upon days.

JungleDisk has also had a couple releases since then and made some improvements.  I also turned the cache down to 100MB, so while its still caching files, its not going to eat away my hard drive while I’m backing stuff up, and they’ve said they are considering an option to turn off the cache completely.

And, while the encryption of my files is less convenient in the sense that I now need JungleDisk to read my S3 files, its not a bad trade off to make considering that my files are more secure.

So my setup is: JungleDisk to interface with S3 account and Robocopy to actually manage the backup process.  Heres my robocopy command line setup too, with some options that work best for me:

set defaults=/MIR /XD “My Music” /XA:H /R:1 /W:2 /Z /V /XO /SEC /COPY:D /XX

robocopy.exe “PATH_TO_BACKUP” “DESTINATION_OF_BACKUP” %defaults% /LOG:LOG_FILE_PATH

/MIR - This tells Robocopy to mirror the file structure
/XD “My Music” - Excludes the “My Music” directory
/XA:H - Excludes hidden files
/R:1 - Allows for 1 retry of failed transfers
/W:2 - When retrying, wait for 2 seconds before starting again
/Z - Copy files in restartable mode to account for network glitches
/V - Turn on verbose logging
/XO - Excludes “older” files, so files that haven’t changed since the last backup won’t be transferred.
/SEC - Copy files with security
/COPY:D - Copy only file data, not other attributes
/XX - Excludes “extras”, this will delete files from S3 if they don’t exist locally

You can find a great list of all robocopy parameters here.

Way back when Netflix released a huge dataset for their competition, I got to thinking about ratings systems like the one Netflix has for movies that you rent.

You can log into Netflix and begin browsing movies and rate movies that that you like or dislike. For instance, I gave “The ‘Burbs” four out of five stars. Based on your ratings, netflix can better recommend movies that you might like.

A similar ratings system exists on the Yahoo Music player. As songs are playing you can rate them 1-4 stars or click on the “no smoking” sign which indicates that the song sucks really bad and you never want to hear it again (of course, there are ways that Yahoo will queue it up for you again, as I’ve heard “drops of jupiter” several times even though it makes my ears bleed).

I like the idea of these ratings systems because I’ve found a lot of music/movies that I like based on the recommendations of Yahoo and Netflix. What I don’t like about these ratings systems  is that they have too many options.

I nearly always end up rating songs at 3 out of 4 stars on Yahoo (if I rate it at all), and I think this is my thought process:

1. I like this song, what should I rate it?
2. Well, its not the BEST song ever, so its not a 4
3. On the other hand, I like the song, and a 2 seems like a bad rating
4. I’ll go with a 3

At least to me, theres no reason to have any other option besides “Thumbs Up” or “Thumbs Down”, either I like the movie/song or I don’t. Digg got this part right on their site, you either “digg” a story, or you don’t, simple as that.  I would think that simplifying the ratings system would make the recommendation engine a little easier too, since they don’t have to account for different “levels” of how much you like something.

As an aside, when I first started using the Yahoo music engine, I found that there was some way to change the rating system from a 4 star to a 0 - 100 system. 0 - 100??!?!  “Gosh, I don’t know, is this song a 33 or a 34?”. Give me a break.

I saw this headline come through my rss reader yesterday, its from a local news channel’s web site:

“Mpls. Workers Brave Frigid Cold, Some With Coats” (link to story)

This seems a little disconcerting.

Only SOME people were wearing coats? With temperatures below zero, I would guess that EVERYONE who went outside had a coat.  And, I’m willing to bet there was a high percentage of people with hats and gloves or mittens too, you just wouldn’t know it from the headline.

It did mention that some people also drank warm liquids to fight the cold, which is comforting to know. We get this insightful quote about the benefits of hot drinks:

“It works. It helps.”

So there you have it - if you’re cold, try a hot drink, it will work, it will help.

So I saw today on Slashdot that Netflix is offering streaming movies. It looks pretty cool because you don’t have to pay anything extra to get 18 hours of streaming content a month. You can watch whatever you want, whenever you want, until you hit 18 hours. I’m not sure if you can pay more to get more monthly hours or not, I’m sure that will be an option eventually.

This has, more than anything else, made me want to get some device (Apple TV, Xbox, etc) that will allow me to stream movies from my computer to the TV, preferably over a wireless connection. In the past (before today), I wasn’t all that interested in these solutions because I don’t have a huge library of movies, nor do I want to maintain a huge collection. Now that Netflix can host my collection for me, connecting my TV to my computer makes a lot more sense.

A couple questions still remain though.

• Is it possible to stream a netflix movie from my computer to the TV?  There is a special movie player that you have to use and I don’t know enough about streaming content from the computer to the TV to answer this.
• What’s the maximum quality we can achieve?  DVD?  HD?  It says that Netflix will detect your connection speed to determine the quality it can give you, but I would be willing to wait 10 - 15 minutes to buffer a higher quality movie.
• What’s the selection like?  The screencast above mentions that Netflix currently has ~1000 titles available.  I’m assuming that will continue to grow.

So we’ll see what happens, I’m not sure what it would cost to get a decent streaming solution from my computer to my TV, but hopefully I can get something eventually.

This is a pretty pessimistic outlook on an install - I suppose it is nice to see that they aren’t beating around the bush either - “….several minutes to several hours….”, yikes.

I recently needed to do a sql query in MySql that selected a bunch of items by their ID, the query was something like this:

SELECT * FROM Customer
WHERE CustomerID IN (502, 506, 398)

I just wanted some customers (more like 50, not just 3), but I also wanted them sorted in the exact order that I specified their ID’s by, so I wanted customer 502, then 506, and then 398, in that order.

It turns out theres a feature in MySql that allows you to specify an arbitrary sort order in a query, it looks like this:

SELECT * FROM Customer
WHERE CustomerID IN (502, 506, 398)
ORDER BY FIELD(CustomerID, 502, 506, 398)

You simply specify the field that you want to sort on, and then give the sort order of the items you want to use. You don’t even have to be selecting on the field that you order by, so this would also work:

SELECT FirstName, LastName FROM Customer
WHERE CustomerID IN (502, 506, 398)
ORDER BY FIELD(CustomerID, 502, 506, 398)

Here, I’m not including CustomerID in the results, but I can still use it in my custom sort.

We just finished up the rest of our tasks for the next release of my project at work on Friday, and it feels great to be done right before getting a couple days off for the holidays.

I recently got my Amazon S3 account setup so that I can use S3 to backup my computers at home. I initially was trying to use a combination of JungleDisk, WebDrive, and robocopy to manage the backups. JungleDisk is the interface between S3 and windows, WebDrive allows you to map your S3 account to a network drive in windows, and robocopy is what I use to mirror my local directory structure on S3.

This solution was “ok”, not great cause it was complicated, but it got the job done. WebDrive also is not free, so it would cost money after the trial period ran up (~$50 I think).

Then, JungleDisk released a version that included built-in drive mapping support, which was great because that made the solution free beceause WebDrive was now out of the picture.

I was having some trouble though, because while I was testing my backups, I noticed that files were being uploaded each time I would backup, even though the files hadn’t changed. I came across this post by Scott Bellware about his switch from JungleDisk to S3Drive and I think I found the cause of my issues, the JungleDisk cache.

So I’ve now switched to S3Drive which also has built-in drive mapping support and no cache so its really a “WYSIWYG” way to manage your S3 account; you can be sure that what you see is actually whats on Amazon’s servers. While I liked the encryption feature in JungleDisk, I agree with Scott that I like having a simple mirror of my directory structure rather than encrypted files that need JungleDisk to function.
I’ve got my new laptop backed up on S3 as well as the server in the basement, all thats left is the desktop in the office, which is going to be ~25 GB when you put in all the pictures that Erin’s got on there. I’m still debating whether or not to backup all of our mp3’s since that will add a lot of space, but its so cheap its tempting.

We’re leaving for my parent’s house in about an hour, so I’m going to start the backup when we leave, so hopefully its done by the time we get back tomorrow night.

The project that I’m on has a biannual release schedule, so we’re in crunch mode to hit the next release thats coming up in the next couple weeks. I’ve been working late, but some other guys have been working even later; I’ve gotten build emails at midnight this week.

I just wanted to relay a funny thing that happened this week. I was working on a file that someone else was working on too, so I just manually made the file writable (we use source safe) and made my modifications. After I got his changes, I used WinMerge to merge our changes together and checked the result in.

Of course, I screwed up the merge and overwrote his changes. He emailed me later and said “Hey! You overwrote my code!”, so I apologized and merged the code correctly and got it all squared away.

After explaining to him what had happened, that I thought I had merged it correctly, he replied with this email:

“The best WinMerge is in your head ”

That’s so true.

Since I’ve been reading Just for Fun, about Linus Torvalds, I’ve gotten more and more interested in using Linux as a desktop OS. I’ve tried Linux before, but I usually grabbed some old, shitty computer to install it on, and I never really forced myself to use it.

This time, I’ve taken a leap though. I have a second hard drive on my computer that has (had) Vista RC1 installed on it that I never used ’cause Vista wouldn’t recognize my wireless card and without the internet a computer is pretty much worthless.

So today I installed Ubuntu 6.10 on the second hard drive and I’m writing this post from my new linux install
The only downside so far is that it’s a pain in the ass for Erin to be using the computer and then having to restart the machine to use linux, I wish there was a way to have both OS’s running so I could switch out quickly. As it stands, I have to restart and press F11 when I boot up to manually change the boot order to boot from the linux hard drive when I want to log in.

I’ll spend some more time tomorrow getting all the necessary stuff installed (Apache, MySql, etc.), and I’ve already got MonoDevelop but I’m still looking for a good PHP IDE, one that has intellisense preferably, but for now I’m going out to try out one of Surly’s newer beers, the Darkness.

I’ve been reading this book lately, Just for Fun: The Story of an Accidental Revolutionary. It’s a first person account of the creation of Linux, as told by Linus Torvalds to David Diamond.  So far it’s been very interesting, I like the way they’ve mixed the technical details of developing Linux with some personal background on Linus’s life as well.  Hearing about the viral growth of Linux is really cool too, it’s funny how it makes you want to go and download a copy and start using Linux as your desktop OS.