I've probably written the following line a hundred times: "The Xfce desktop didn't seem any quicker than GNOME." But that was all about mousing around in the menus, not so much how quickly applications load. And after running the Xfce-based Xubuntu, Vector and ZenWalk, as well as running Slackware with Xfce, I decided to try it in Debian.
Now my $15 Laptop (233 MHz Pentium processor, 64 MB) has been running Debian with Fluxbox since I got it running -- I started with a console system, then added X and Fluxbox -- so I have no idea how it would respond on the heavier-than-Flux Xfce, but for my 1 GHz VIA C3 system (which I suspect is running at much less than the full 1 GHz for some reason), Xfce is the way to go.
That said, I finally found a distro on which KDE doesn't run like sludge on my test PC.
more here