Some people know that I’m writing a web-based RSS reader in Ruby on Rails. One of the very stupid details that I have been pontificating is favicons. It seems that Yahoo’s reader just blindly links to the server’s favicon URL by default. I’m not sure that Google’s Reader does anything useful with them. In my opinion, I think it’s best to just cache them on the RSS Reader’s server(s) because the originating site may be unavailable. This got me to thinking about what if some HTML contains an img tag pointing to an external server. Should my server cache those thumbnails as well? The offsite image thing could burn through more of my bandwidth, however, it would safeguard against external server availability.

Another thing I was thinking of is somehow managing Livejournal cookies on my server. It kind of gets to me that I can’t read all of the secret LJ posts with other RSS readers. The secret posts are usually the good stuff that injects much needed drama into my life. Basically, my Livejournal friends page is my RSS reader right now for this reason. I use NetNewsWire on OS X too, but its not really doing what I want. This is really the first time that I’ve ever been annoyed so much with current software that I’ve tried to roll my own solution.

I just had an idea tonight for a way to automatically download and build a source release tar ball from an open source project. I call it Sourcecasting. I’m sure this idea can’t be novel because it merely extends the ideas of podcasting and photocasting, but I was thinking that it might be convenient for project, distro and rpm maintainers particularly if open source projects had a unified way of releasing their latest versions. So, why not create an RSS feed with links to the last 10 or so source tar ball releases e.g. a FOSS project Sourcecast so to speak.

Most FOSS projects have an ftp site and some methodical way of naming files, and this has been working pretty well since the grand experiment had begun. There are some problems with this because it does require quite a bit of infoware (custom perl/python, etc) to handle these in any kind automated way. A Sourcecast would really be a convenient method for everyone compiling or releasing source code tar balls in whatever capacity.

There are several advantages to Sourcecasting over just putting a download link to Sourceforge and using a weblog to promote your project. First, this method would systematically let everyone know where the last 10 or so release can be downloaded. Next, a Sourcecast RSS feed could even be used to load balance or to perform a phased release using GeoIPFree (e.g. incrementally roll out your release on a per country basis). You could even go in finer granularity if you had state and ZIP Code IP address databases (which cost $$$$). Another interesting spin on load balancing this way is that you could even write some javascript to (initially, not for every load) calculate the fastest download link for your site, and then send the latencies back to the server via SOAP or XMLHTTPRequest, the server could then create a custom RSS feed specifically for your location. This could then be updated on an as needed basis. For huge source code releases, you could even use a torrent to mitigate bandwidth charges. Another large advantage to this method would be that all open source projects now have a machine parsable release strategy, and this could really improve the productivity of distro and package maintainers (Red Hat are you listening?). Not only that, but it’d be useful to add Sourcecasts to an RSS Reader just to see when your favorite project has released new version.

I know it’s kind of silly to expect everyone to start doing this, but there are some backdoors. Probably the easiest way to do this is for free would be for the Apache Foundataion add an automatic RSS feed to mod_autoindex. This way, Sourcecasting is essentially painless and would come for free if you ran Apache’s httpd. I think this could really be a huge boon to productivity for FOSS, if implemented correctly. Also, I wouldn’t be surprised if some projects are already doing this in one form or another, but I haven’t seen any mention of it on Google. So, at the risk of sounding like a sanctimonius dolt, I present this idea as my own.

Microsoft has announced a draft proposal for SSE today. I actually like the proposal a lot. It seems to accomplish some very sophisticated means of aggragating information and syncing schedules and anything that you need to share. Something really needs to be done, as the current crop of tools that are scratching this itch are woefully inadequate and require too much intervention. I think that SSE is a step in the right direction. I sent the following email to their mailing list (which appears to be a fairly moderated system even if you’ve subscribed).

It seems that in a star topology, SSE would somewhat approximate what Exchange and Lotus Notes seem to accomplish as pertains to scheduling. If you used SSE in a mesh configuration, it would seem to require a peer-to-peer protocol to negotiate communication between links that use network address translation. Would Avalanche technology or another companion technology be used to negotiate mesh networked nodes? If so, why would you need to poll in the sense that Outlook and Lotus polls a server for mail and schedule updates? If SSE requires a protocol with this additional sophistication, why not use one that would allow broadcasting update metadata to all the nodes that are active on the mesh at the instant of a subcribed feed update from a publisher? It would seem that standard would even allow for this as it is not specified that you must poll outside of the FAQ. If you sent an XMLRPC or SOAP broadcast over Avalanche wouldn’t this eliminate lots of unnecessary poll packets if you scale SSE to a distributed mesh topology? Or am I missing something here?

Also, it would seem that non-endpoint feed nodes that are sending SSE feed data to or from a node other than itself could be fairly insecure when not transmitted using a secure application protocol. If such a protocol were used, it could be fairly wide security hole. Effectively, this would open up an ad-hoc mesh network where my node would be unaware of what data it is sending if it were protected via key pairs and unencrypted at the end points. This seems to be kind of a Catch 22 situation. If you don’t encrypt it, I can put my ethernet card into promiscuous mode, and gather all the packets from the phy and gain access to unencrypted xml data. If you do encrypt it, such a network would be an anonymized bittorrent and nearly like what Freenet attempts to do by distributing and anonymizing all data. I guess I’m not seeing how SSE can be implemented securely and reliably using an existing protocol. It wouldn’t take much to uuencode binary data and transmit it over a protocol intended for text-only similar to what has happened to nntp.

I think what is missing in the SSE is allowing for some type of relevance metadata. Google seems to have implemented an interesting XML Sitemap Protocol. Within Sitemap, there is a field for priority. I feel that this is a much overlooked notion in RSS Aggregators on the whole. Abandoning the schedule syncing example, say you’re syncing news feeds between SSE clients. This is almost like bloxor with automatic syncing of your opml blogroll directly or via a mesh network ala Bittorrent. Within Sitemap, the site publisher can specify what is the most important data that it serves via the priority field. If SSE were to include some type of optional relevance or priority information, then it would be possible for the users to specify what data is relevant to them to meet their own needs. This data could then be harvested, and used to benefit everyone using the network. Consider recent startups like Del.icio.us, Digg, Stumbleupon, Reddit, and Bloglines. These social bookmarking sites are a powerful way to find out the goodness metadata of news and articles based on how many people like or dislike the information. So, if you’re syncing lots of SSE feeds back-and-forth, each node effectively would have access to this information about the information. The existence of a link in an RSS feed would roughly be equivalent to what Google refers to as Pagerank. This type of information processing would be better suited in a star topology, but could be extended to a mesh if all priorities were made visible to non-endpoints and thereby eliminate server-side processing. Using priorities would also enable the implementation of automatic server-side priority calculation software (I’m currently implementing this in Ruby). This would enable each web server to automatically use popularity as a way to help the clients adjust the ordering of the RSS feed data that is being offered. In a mesh network, you could essentially farm out the server side processing that’s used to calculate whether or not something is of interest to a particular node.

You may be wondering, “WHY DO YOU CARE ABOUT THIS!!!?”. The only way that I can explain it is that I want to control the way that I consume information. It’s some sick twisted plan that I’m cooking up to automatically ignore irrelevance (mitya000 not included, because that’s some good irrelevance), and pay attention to important events, automatically.

I added a new quotes feed that shows the last 15 quotes that were added to my site. Rather than have author names appear as titles, I chose to use truncated quotes, and put the full quote and author in the decription. You can subscribe to this feed here .

I just added an RSS 2.0 feed with proper date formatting (this works in bloxor). If you need such a thing, you know where to look.

I added a subject field into my rss feed at one point, but it didn’t work with the dtd for rss 1.0. So, I commented it out with a #, apparently that is bad xml foo or something, and it wasn’t parsing with Firefox Deer Park Alpha 1 (it did parse under Firefox 1.0x). It’s working now. Nothing to see here. Move along now, move along.

Some geezah managed to put up a Homestar Runner RSS Feed here . It’s pretty neat how it works by using perl to screen scrape the site periodically.

I’ve tried news aggregators like the Sage plugin for Firefox, but I didn’t really care for how it displayed the feeds. So, I just pootled along using my Firefox Live Bookmarks as my “aggregator”. However, after Yahoo! started allowing custom RSS feeds to be added , I setup an awesome page of feeds from my Live Bookmark collection. Now, mind you Yahoo! does have advertisements, but if you use the AdBlock Firefox-plugin with Proximitron, their detriment is rather marginal. The only thing I’m kind of wanting now is for the page to sort the feeds in the order of whose been updated last.