Ben Reichelt’s Weblog

Awhile back I was thinking of making a web based rss aggregator like Bloglines. I thought that the concept was pretty simple, so it shouldn’t be that hard to implement, but there are a couple of trouble areas that you run into when developing an rss aggregator.

• Parsing the feeds.  This can get really hairy at times because you want to make sure that you can parse all the feeds that your users are subscribing too, even though those feeds may not be valid rss/atom feeds.  In order to account for all the different feeds, you need a really robust feed parser, which can be a nightmare to write yourself, and I didn’t really find a good .net library that would handle this for me.  I looked into grabbing the code from RSS Bandit to do it, and I also looked at using the Universal Feed Parser, which is written in Python, and using Iron Python to incorporate that.
• Scaling.  This is huge and probably the biggest problem.  The aggregator that I ended up writing works great for valid rss feeds, and as long as I’m the only one using it :)  It can run on my single desktop just fine, but I suspect that if you loaded up any more than 30 - 40 users, the thing would crash and burn, especially if those users all had a nontrivial number of feeds that they subscribed to. The scalability problem has been well documented as feedlounge has been growing. They have been really transparent about the issues they’ve been hitting as they ramp up their service, and scaling has been their biggest obstacle.

Another idea that I had to try and alleviate the parsing problem, was to use Bloglines as sort of a proxy.  Bloglines republishes the feeds that their users are subscribed to, and the feeds that they publish are guaranteed to be valid rss feeds, so I wouldn’t have to worry about parsing any invalid feeds, which makes life a lot easier.

In the end, I realized that I wouldn’t be able to afford the hardware that would be necessary to run a web based aggregator, so the project died off, but it was an interesting problem to work on as there are some different ways to implement the aggregator, so making some design choices was fun.

I have a class that I want to track the IsDirty property on, so that I know if the object has been modified or not, so that when the object is saved to the database I can simply ignore the save if no changes have been made.  If a change has been made (IsDirty == true), then I want to go ahead and update the database and set IsDirty back to false.  However, what I don’t want, is to let anyone modify the IsDirty property, I would rather make it readonly so that people can’t mess with it.

The problem that I’m having is how to implement this, because the object doesn’t handle its own database persistance.  The object gets handed off to another class that will do the appropriate database operations, so the object is little more than a strongly typed DTO.  So I want to allow that other class, lets call it the Persistor, to be able to set the DTO’s IsDirty property back to false, but nobody outside of the Persistor and the DTO should be able to modify the property.

I can only think of one way to implement a solution that behaves how I want:

• Make the property read/write but inside the setter, do a runtime check of the class that is attempting to set the property, and if its not of a certain type, throw an exception.

Frankly, this solution sucks because its going to take a performance hit each time I set IsDirty from the reflection of checking the caller, and it sucks because there’s no compile time checking going on.  There’s got to be a better way to implement this, but I can’t think of anything, other than changing the overall architecture.  Thoughts?

My buddy Mike just pointed out to me that Netgear has a new Skype phone, and guess what?  Its based on WiFi, so you can bring it all over the place.  The software is hosted on the phone, so wherever you can get an internet connection, you can use your Skype phone.  What this means is that the first piece of the puzzle has been fulfilled, the next piece we’re watching for is readily available wireless internet access.

I had a need to sort collections of objects that were stored in ArrayLists. There were about 30 different business objects that could potentially be used, and none of them had a custom collection class, ArrayLists were always used. None of the objects implemented IComparable, and I didn’t want to write an IComparer for each object type either. So I came up with this PropertySorter class that contains a string, the name of the property, and a bool to determine if we are sorting ascending or descending. The PropertySorter implements IComparer and it uses reflection to find the given property on each object that it compares, so now I can use this same class for all my comparisons, I just need to supply it with the name of the property that I want to sort on.

What I realized I needed later on was the ability to sort on multiple properties, corresponding to the SQL syntax “ORDER BY FirstName, LastName”  which returns your results sorted by the firstname, and then last name in a hierarchical fashion. So I changed the string property on the PropertySorter to an array of strings and implementing the cascading sort was actually rather easy.  This class works like a charm, I love it! I don’t care that it uses reflection, it saved me a BUNCH of time and my web pages don’t seem the least bit slower. The only restriction that the PropertySorter has is that the property on which you want to sort has to implement IComparable, otherwise it wouldn’t know how to determine which was less than, greater than, or equal to, this is also no problem as I only want to sort on strings, numbers, and dates anyways. Here is the PropertySorter class:

I’ve been using Notepad2 for quite some time now, after seeing it on Scott Hanselman’s list of useful tools, and I love it. The one feature that it didn’t have, however, was tabs. Whenever I have multiple text files, xml files, config files, etc. open it clutters up my taskbar, so I wanted something that could handle them all in one window, with tabs. This is how I stumbled across the Programmers Notepad.

This thing is great, its got all the stuff that Notepad2 has, and its got tabbed windows. Excellent.  Still loads up fast too, cause the last thing you want is for your text editor to seem like Adobe products, heh.

I hadn’t seen this article by Dave Winer until just recently when it was linked to by someone else (can’t recall who). The style of aggregation that Winer describes here is exactly the reason that I like using Bloglines so much. Bloglines just gives you all the content in one shot; click on the feed, and you get all the new items lined up for you. I dont want to have to click my mouse, or the down arrow, to go to each new item, not when I can scan the items myself in a much faster way. I would say that I read about 30% of what comes into my aggregator each day, the other stuff is just skimmed until I know that I’m not interested. Its too slow to work this way when you have an Outlook style aggregator that makes you look at each item individually. The human brain can filter the items much faster and better than any aggregator can, and by using that you can subscribe to a lot more feeds while reading less and making better use of your time.

I do, however, like to have a list of my subscriptions on the side, rather than just getting everything in a true ‘stream’ style. I don’t want the posts from all of my subscriptions mingling together, I lose my context that way. When I click on a subscription with new items, I can prepare myself for the content that I’m going to be getting. forgetfoo.com is going to be giving me different stuff that espn.com which will be different from Adam Bosworth, and it would throw me off to get items from these sources intermixed.

This is the way that I like to read the web each morning, and I really don’t see myself ever using any other style.

The Sql Server madness continues at work. Here’s a little tip on how to reset the identity column of a table if its set to increment by one for each row thats inserted, without truncating the entire table.

If you run the following command at the sql prompt, you can reset the identity field:

DBCC CHECKIDENT(‘myTableName’, RESEED, 0)

This will set the identity field to 0 on the table ‘myTableName’.  One caveat with this is that if you have rows in that table, it will allow you to reseed the table to an existing ID number.  For instance, if you have a Users table, with a UserID auto number field, and there is a User with UserID = 10, you can reseed the table to 9, and Sql Server will let you.  However, when you try to insert another row, you’re going to get an error because the UserID 10 already exists.

On an internal mailing list a little while back, someone was having trouble with the Sharepoint installation. When they would go to install Sharepoint, they would enter the registration code and whatever other information that was required, and then the next screen would promptly start to UNinstall Sharepoint!

So, about 30 minutes later, the person doing the install emailed again, saying he had found the problem.  Apparently the user account that they were using for the Sharepoint database had a dash “-“ as the first character of the password, so when the username and password were passed on the command line for the setup, it was actually interpreted as another parameter to the command line program! I just found that interesting, of all things to cause problems, sheesh……….

This is just a tip about using SqlCommand objects with SqlParameters and null database values. Here’s one way to add a parameter to a command object:

SqlParameter param = sqlCmd.Parameters.Add(”@FirstName”, SqlDbType.VarChar);
param.Value = ‘ben’;

This adds a parameter named ‘FirstName’ with a datatype of ‘VarChar’ and a value of ‘ben’ to my SqlCommand.  Now, if you wanted to make the value of the parameter a null value, this is what I typically do:

SqlParameter param = sqlCmd.Parameters.Add(“@FirstName”, SqlDbType.VarChar);
param.Value = null;

Since the Value property of a parameter is of type System.Object, you can simply set it to null.  This is wrong.  Don’t do it.  You’ll get exceptions about missing parameters.  I ALWAYS forget this and I’m ALWAYS confused by the errors until I remember this.  If you set the parameter value to null, the SqlCommand must simply exclude the parameter when it executes, which would cause the missing parameter error.  Instead, this is how you set the value of a parameter to null:

SqlParameter param = sqlCmd.Parameters.Add(“@FirstName”, SqlDbType.VarChar);
param.IsNullable = true;
param.Value = System.DBNull.Value;

There is a special class ‘DBNull’ that is used to represent NULL values, so if you do it like the example above, you should be all set.

I’m trying out Resharper as I’ve heard many good things about it for quite some time. I’m about 2 weeks into my 30 day trial, and I do love the refactoring options it gives me, I especially love the feature where I can create a method on the fly and then Resharper will add the stub for me, very sweet.

One thing that annoys the hell out of me though, is that it doesn’t remember my preferences correctly. Every time I open VS (2003), I have to go into the Resharper options and tell it to use the VS Intellisense instead of the Resharper Intellisense. Has anyone else run into this, it really bugs me…………….