Xarg blog Archive

What is RSS? A nice introduction to the various flavors of RSS by Mark Pilgrim at XML.com.

I finally fixed the rl_returnz code to do the correct output encoding (so Val d'Isère and any other iso-8859-1 style data should now display correctly). The patched code is available for the adventerous.

I spent a couple hours looking into why the cc_users view performs so poorly on postgresql. I narrowed it down to something in the genetic query optimizer but had not really been able to figure out how to get it to go fast no matter how hard I tried. I asked for help on on irc at #postgresql which was not terribly productive but I wrote it all up, asked on the pgsql-performance mailing list, and had the answer in less than an hour from Tom Lane who is the guy responsible for the optimizer. It turns out I was sort of on the wrong track since it was not really the generic query optimizer rather it's that the runtime parameter GEQO_THRESHOLD is also used in a heuristic to decide when to flatten subselects or not, but the answer is the same as I came up with which is to increase GEQO_THRESHOLD.

It always seemed like a kind of cliche when people claimed that's how things happen with open source and given the signal to noise in a lot of places I was not terribly hopeful I was going to get a real solution but I did. Now I feel like I could be in some sort of testimonial commercial about open source. Certainly compared to trying to figure out why my oracle queries were so horrible this is a refreshing experience.

I found someone who feels more or less the same as I do about Netscape 4. From Bitworking:

I really don't care about people using Netscape 4. As a matter of fact if I could find a sequence of tags that would cause NN4 to do nasty stuff like erase your hard-drive or shoot lightning out of the screen when viewing this site I would gleefully add them. Ok, maybe that goes a little beyond a don't care attitude.

Actually, I kind of feel the same way about IE and fortunately this is in fact possible in IE (the erasing the hard-drive bit, not sure about the lightning). There are half a dozen things you have to do to support IE that are annoying and stupid and it's hard to understand why they have not been fixed. The bug with reloading things sent with gzip encoding is one and the failure to properly handle transparent PNGs is another. The arrows in my context bar and the site logo are both transparent PNGs that look great in Mozilla but have white backgrounds in IE. I have decided to build my site using things I wish worked right everywhere and eventually hopefully when most people's browsers are standards compliant it will look about the same for everyone. Until then, people using IE will assume I am a knucklehead and I will continue to loathe IE.

There is a great article by Andre Torrez about building sites:

That's what's got me so bothered about people musing in their weblogs about projects they'd like to do. Stop talking about it an just build it. Don't make it too complicated. Don't spend so much time planning on events that will never happen. Programmers, good programmers, are known for over-engineering to save time later down the road. The problem is that you can over-engineer yourself out of wanting to do the site.
The only thing special about the code is that it was written. The only thing special about the sites are the users.

The OpenACS community suffers from a surfeit of ideas and not enough sites using the current code. I am certainly as guilty as anyone of talking about things but not acting, although I have made a real effort to get the code used on this site both out in the open and back in to the OpenACS code base. It would be great if more people working on the toolkit would try to run sites which actually provided some services to other people (and I fully intend to follow through with this, rather than simply write about it).

We have enough good ideas already. We need better documentation, more testing, and more bug fixes. But more than anything else we need people actually running services on OpenACS doing what it can already do and developing things further based on meeting peoples real needs. Launching openacs.org using 4.x has done more for the quality and functionality of OpenACS in a short period time than just about anything else we have tried. Running this site (with almost no services other than photos of my kids and a cvs log browser) has made me focus on problems with the toolkit I would have overlooked otherwise.

Interesting post on collective ownership and what has and has not worked for the Apache Jakarta Project by Andy Oliver

community code is a fundemental principal of community-based software development. If the sources are owned by the community and can be changed by the community then progress happens much faster than if each section of code depends on one person.

He talks a lot about the need for trust and respect which I agree with, although I think removing all the author tags from the code is a pretty extreme way to make the point.

I think with the right kinds of tools (an easy way to browse and back out commits, good tests, good notifications, etc) collective ownership is a huge win. That was my motivation in writing the cvs log browser and hopefully at some point with OpenACS our community development tools will be good enough that a dialogue centered on concrete action will be the most natural way to do things (see Sam Ruby and Rael Dornfest and Andy Oliver again for more comments on code, community, and decision making).

I installed rl_returnz on the server so pages are now served compressed for browsers that support it. I tried it in IE5.5, NN4, and mozilla 1.2.1 and things worked fine. It should make some of the bigger pages like the cvs log browser snappier for slow connections.

There is a great interview with Martin Fowler at artima.com. One thing in particular that caught my interest was his views on code ownership (and they are the same as mine).

Martin Fowler:

Weak code ownership and refactoring are OK. Collective code ownership and refactoring are OK. But strong code ownership and refactoring are a right pain in the butt, because a lot of the refactorings you want to make you can't make. You can't make the refactorings, because you can't go into the calling code and make the necessary updates there. That's why strong code ownership doesn't go well with refactoring, but weak code ownership works fine with refactoring.

It's exciting to see the uptake of the RSS 2.0 Blog Archive format described by Dave Winer. Theres are namespaces defined for Radio and Movable Type and there are several blog browsers being written.

From looking at the server logs I see more and more services which are consuming blogs and hopefully doing something interesting with them. Providing full archives in an easily consumed format should help that along. Blogdex and daypop are cool but even cooler are the services that will take your archive and help you explore the things you are already thinking about like technorati and blogstreet. I would like to see a service take your browsing history, let you score your interest in the pages you visited, and generate a personalized syndicated feed. (something like the Amazon "Page You made" page only with ideas rather than products).

CBC News: Spiders weave huge natural wonder in B.C.

PRINCE GEORGE, B.C. - A biology professor in northern British Columbia has spotted a clover field crawling with spiders. Brian Thair of the College of New Caledonia in Prince George said he saw a silky, white web stretching 60 acres across a field.

I would have loved to see this. When I went to New Zealand there were places that were just covered in spiderwebs but this is even more extreme. I really like the idea of giant spiderwebs.

There is an interview with Maryam Mohit of Amazon.com up at goodexperience.com about usability and customer centered design.

I particularly liked this since it captures what I think is exciting about building things on the web:

For us, it's a combination of listening really hard to customers, and innovating on their behalf. For example, quite awhile ago we developed the "similarities" feature - the one that says "people who bought this also bought that." In focus groups, no customer ever specifically requested that feature. But if you listened to customers talk about how they buy things, they'd say, my friend bought this, and I like what they like. In other words, they get recommendations from people they trust. There was a cognitive leap, based on those comments, to realizing that we could create something like that based on the data we had. That's an example where there was a need expressed by customers, but the innovation was taking that general need and making the leap to a technology that meets that need in a new way.

She also mentions some of their in jokes like the butterfly ballot joke and the current Customers who wear clothes also shop for: clean underwear bit which shows up promoting clothes purchases in the books section. I had seen the tag line before and wondered if it was intentional or just another microcontent mistake...

I tried out AmphetaDesk...It's cool and was trivial to install on redhat 7.3. Sort of a light weight version of radio userland's aggregator. Since it's in perl I can add a "post" button to the stories I think it would be really work the same (less some intelligence about when to pull files down).

Also, I wrote a bookmarklet for submitting stories to Lars-Blogger as well (a good idea from MovableType), it take the page title, url and selection (if there is one) and sends it to the entry page. I had to change the code a little to make it work but it works great (under mozilla anyway).

I am not sure how I found myself there but It turns out that Edward Tufte's website actually uses ACS (it looks like acs 3.x).

One thing I found was a nice essay on Gantt charts in his Ask E.T. forum. He has some suggestions for improvement but I found the indictment of current practices the most interesting:

The design of project charts appears to have regressed to Microsoft mediocrity; that is, nothing excellent and nothing completely useless. (Is the reduction of variance around a modest average the consequence of monopoly?) Most of the charts in Google look the same or make the same mistakes: analytically thin, bureaucratic grid prison, not annotated, little quantitative data. The computer Gantt charts, so lightweight and tinker toy, do not appear to have been designed for serious project management.

One thing I noticed though was that he used the same background as me (which I thought was nice), although I think he makes a mistake in using #f00 for visited links and #c00 for unvisited links which is the reverse of the usual suggestion to use a less saturated color for visited links.

The OSTA has released the MultiPhoto/Video Spec which I think is inteded to be the interchange format for CPXe. It would be pretty exciting if this catches on. I was pleased to find that there is an LGPL'd MPV SDK which it looks like HP has contributed. I am going to take a look at it and see if it makes sense to have photo-album be able to generate MPV data sets.

Here is a something from the MPV Overview

Executive Summary

The Optical Storage Technology Association (OSTA) has developed an open specification available at no cost that enables the playback and exchange of collections of photo-video content. The format, called MultiPhoto/Video (MPV), allows users to create and exchange multimedia albums and playlists of content from today's consumer digital cameras, such as still images, stills with audio, still burst sequences, and video clips. The goal is for MPV to be widely produced and consumed by digital cameras, retail picture disc services, imaging software, internet services, DVD players, and other devices.

MPV is also a part of the Common Picture Exchange Initiative (CPXe) that is underway within the International Imaging Industry Association (I3A). CPXe will enable easy transmission and printing of digital images between digital cameras, PCs, desktop software, Internet services, photo kiosks, digital minilabs and photofinishers.

It's on /. so everyone has probably seen it but Microsoft suggests you should not trust them. The relevant bit:

What steps could I follow to prevent the control from being silently re-introduced onto my system?

The simplest way is to make sure you have no trusted publishers, including Microsoft.

I don't hate Microsoft (well, maybe a little...) but I do hate a lot of the web technology they backed. ActiveX was and is evil and this makes it pretty clear why. Also, it's cool there is a word for what I feel :)

I think anyone who works on bug-tracker should read Call Center: Profit or Loss? by Bruce Tognazzini in his AskTog column. It's a funny article and he makes the very good point that call center support can be a real material expense and that properly handled can be both much cheaper and improve the companies products and support.

You have to believe that most call centers have a support DB to help field calls (and the ones that don't probably cost companies even more money) and it is a small step to turn tie that into a ticketing system and generate real dollar figure costs for helping people resolve problems with your products. Which should ultimately direct you to how to resolve the problems at the source: your product's design and documentation.

I think the improvements in consumer class PC hardware installation documents are the product of this feedback loop (the simple install poster for hard-drives certainly has that feel).

I found the report How Do People Evaluate a Web Site's Credibility? Results from a Large Study to be pretty interesting. The headline result, "design matters more than anything else" should not be a huge surprise. I guess the question in my mind is does credibility matter?

It reminds me of the article I read (which sadly I can no longer find) about how a large site tested various different banner ads and found the most grotesque ones had the highest click through rates. The designers were horrified by it and ads like that certainly can't do much for credibility but that was what drove revenues at the time. Maybe now that people are increasingly blind to banners it is credibility and brand awareness that matter more than simply screaming for attention.

I put both packages on the code page for download. I wanted to clean them up some more but realistically I don't have the time to do it right now and maybe they will catch someone else's fancy and they will fix them for me (which is not to say they don't work since both are running on this site, they just could do a lot more).

I would like both to produce rss files, the photo album clipboarding could be better, it would be nice if they were searchable, it would be nice if the cvs log browser updated incrementally to the DB rather than storing all state on the server, and the list goes on.

I switched the site to use the CSS version of lars-blogger that I worked up. You can see the style switching in action on my blog homepage where there are some different styles to choose from. Just so you know the lars style is just a pale shadow of the Lars-ian original.

I installed Movable Type and played with it some. I liked it a lot and I think if all I was going to do was run a blog I would use it. The templates are nice, the UI straightforward, and it's fast. I really liked the bookmarklet for creating a blog entry.

I installed it under AOLserver running OpenACS and it was pretty easy to get working (although I had to hack the request processor a little to keep rp from hijacking the cgi requests).

This is being discussed on the AOLServer list with some fervor so I thought I would put up the coding standards excerpts from various projects that I had collected when I looked into what other large scale projects did.

the CVS log browser I wrote is up and running...take a look. The colors for the last update were snarfed from sourceforge's bug tracker. It's a little bit of a hog though. My aolserver process now takes 180mb instead of 40 (and as far as I can tell, only 25mb of the 140 are really data, not sure what happened to the rest).

I put up some butterfly pictures that I took at the London Butterfly House in Syon Park. I did not manage to get a picture of a blue morpho (there was one fluttering around) but I did get a few I liked. The pictures were all taken hand-held with flash and would have turned out a lot better if I had a diffuser and flash bracket. All in all, though, I was pretty happy with them. Compared to taking film based pictures with the same lens, no focal length multiplier, and a flash sync of 1/250s this was much easier.

I added a clipboard for users to collect images up for printing and tied to in to the shutterfly.com C4P system. It was actually pretty easy to do and the photo-album stuff is almost to the point where it's usable. I still have to do some "over the shoulder" usability testing and there are some shortcomings with the print ordering that I would like to correct but all in all I am pretty happy with it.

People talk about web services as being great things and it seems like publishig a web interface for places like shutterfly would have a great ROI for them but remarkably very few photo fullfillment vendors seem to have a usable web fullfillment interface. I only found out about C4P from the gallery source code, and I think without knowing it was there you would never find it.

I found fotango here in the UK actually lets you email your images in as attachments which might be an option as well but it seems a little clunky. C4P (which is simply a HTTP POST interface) seems to be a very simple workable answer.

I wrote a something to parse the cvs log file and let you browse the the changes. The neat bit is that it breaks things up into "change sets" which means that you can see the whole set of files modified by a given commit command. It's not foolproof but it does seem to work pretty well.

It links back to SDM patches and bugs and also generates the commands that you would need to rollback a given change set. I will also probably add some statistics and per module browsing and then put it up at http://openacs.org/cvs/

The new openacs.org site has been launched. There are definitely some glitches and some rough edges but having the site running on the current code will mean that things both get fixed much quicker and are tested much more thoroughly (at least the packages that show up on the site). One of the good things about ACS was that photo.net and later arsdigita.com both used it which meant that if there were serious problems in the code they got picked up and fixed quite quickly. Now we finally have that with OpenACS.

I tried out Radio Userland. I like how it all hangs together. It does not do that much but what it does do, it does well. I especially like the News Aggregator. The software and a year's worth of hosting seems like a pretty good deal for $40.

Next on my list to try is Movable Type. I am scared that all that all that perl might hurt my brain.

Ideally what I want to do is set up a weblogs.openacs.org so that we (the openacs.org community) would have a better means of coordinating ideas, proposals, and ongoing projects. I would also like to make it a platform where we could experiment with collaborative filtering, advogato style trust metrics and such. First thing to do though is get the weblog and aggregation tools together to support it.

It's amazing how trying to use your own software gets you to focus on the details. I made all the photo album package work the way I thought I would want it to but then after uploading a couple hundred photos I realized there were real shortcomings in the UI. Before actually using it (but after fairly exhaustive testing) I would have sworn it was actually pretty usable. Now I know better. Some examples of things I ran into:

• When you bulk edit a page it sends you back to the page you just edited when usually I want to go to the next page.
• It does rotations in the background and because it sends the page with sizes you sometimes get the html out of sync with the thumbnail. I have rotated several images twice because I was not paying attention.
• Long upload times means it is very easy to accidentally interupt an upload in progress (this last one I don't know how to fix).

I find reading server logs relaxing and entertaining. Besides the infinite number of IIS exploits that turn up in the logs there can be other signs evil about. Two entries I noticed recently are illustrative of what you can find if you pay a little bit of attention...

134.157.24.10 - - [13/Oct/2002:12:08:01 -0400] 
    "POST /cgi-bin/formmail.pl HTTP/1.0" 404 549 "http://j3ff.com" 
    "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90)  
     via proxy gateway  CERN-HTTPD/3.0 libwww/2.17"

Here is someone trying to send spam via a (non-existent in my case) buggy form mail script. The interesting bit is that they have trawled the whois records to pick out the fact that I own j3ff.com as well as xarg.net (presumably because people trying to prevent spamular abuse of their buggy script typically check referrers).

This one, while not really as abusive as the first example is still still spam (although in this case it is spamming the weblog community).

207.253.71.48 - - [24/Oct/2002:00:01:50 -0400] 
    "GET /blog/ HTTP/1.1" 200 19529 "http://referrer.mastodonte.com" 
    "Mastodonte Referrer Advertising"

If you follow the referrer here you find a site that will add "your URL as a referrer in the logs of thousands of weblogs" all for a mere CAN$ 1,500. I guess it is the inevitable result of backlinking and Google's ranking algorithm - another negative externality to fight. Tragic...

I put up my modified version of the photo-album package. I am currently suffering the pain of the A in ADSL as I try to copy about 3GB of photos up to the server. I guess 24 hours is really not that bad...

Soon there will be more family photos than you can shake a stick at. Aunts, Uncles, and Grandparents will no longer be able to pester me about getting copies of all those digital photographs I took :)

I am learning about RDF and RSS which I had never paid much attention to before. I downloaded the dmoz directory in RDF to play with. I think the catagorization and link datasets would make good test data for OpenACS (and PostgreSQL for that matter). Ben Adida pointed me at the Redland RDF parser which conveniently already has a Tcl Binding.

Unfortunately, most of the RSS feeds are not very clean so a strict parser might not really work for what I really want to do (which is build an aggregation service to pull together all the different OpenACS development sites bug trackers, blogs, etc). Since we would be writing the code to generate the RDF data we could just make sure we do it right and just use a strict parser. On the other hand I could see getting feeds from other sources that were not so clean. It's a common problem and it looks like people are thinking about how to attack it.

I liked the Mozilla RDF and XUL Bugzilla example.

I posted the two articles I had been working on How I built this site and My AOLServer has fallen and can't get up so now there is something under writing. Yay. Now I need to get static pages working and add the photo-album module and things will be most of the way there (or at least there will be something under all the tabs).

Well, the rss problem was that rss is not set up properly unless you use the subscr-ae page passing it the secrit impl_id and context to register it with the rss gen service. As for notifications it looks like if you don't first submit as a draft and then publish the notification (and weblog.com ping) don't happen. Well, gradually all this will get ironed out and it will all just work smoothly. Oh, there was also a bug with the way the rss was generated that made blog entries with strings and quotes break the rss generation as well.

neither work right out of the box. It's not clear what the problem is although I think in both cases it is that the default parameters are bad (openacs.org as the sender domain and no defaults for rss files).

I am working on merging the sloanspace changes into dotLRN. Of course, there are conflicts with the work that both Collaboraid and OpenForce have done and not committed. I guess I view the existence of conflicts as a good thing since it means the code is changing.

I rock. Now I am like my hero Lars.