When I told my friend David I was upgrading to Panther, he asked "why?". Good question. Mostly, it was for the better Finder interface, but also because many people reported that 10.3 felt faster. This benchmark site confirms that 10.3 is a bit faster in most areas.
BTW, the new Finder interface (with the sidebar and Exposé) really is quite spiffy. Maybe not $129 spiffy, but much better than the Old Way. And, to my taste, better than any of the Finder-replacement applications that I had tried over the last year.