I started using Linux when the kernel in 0.88 PL1 days (it was amazing on my 386DX 40!) In those days I had an ATI VGA Wonder XL video board in my system. ATI VGA Wonder XL's worked either with the XFree ATI driver or with the XFree VESA driver; except mine. My VGA Wonder had a different clock chip that made it incompatible with XFree. I remember reading through endless guides posted on usenet news groups describing how to patch XFree and how to manually compute the monitor mode lines to make XFree work with this card. The hours I wasted! Of course, because it was all open source, I could patch the source to XFree, and because there was documentation I could compute the mode lines. Go open source.
In the late nineties, I was working as a systems administrator. I installed Linux on some Zenith workstations. They had (almost) generic video cards. XFree failed to work on them without some more configuration file tweaking. To make it worse, the application we used required OSF/Motif libraries, which were not free (even as in beer) in those days. The commercial libraries required specific versions of XFree, which were unstable with the video cards. The best we could reliable do was to put them in VESA mode and live with 800x600x8 displays.
In that same era I picked up a Dell Latitude laptop with a NeoMagic video card. That was the one and only laptop which I had no driver issues with. There was a NeoMagic video driver already there, and it happened to work with Linux.
Ever since then, I've struggled with video card issues. My newer Dell laptops have all had great video support under Windows, but terrible support under Linux. Even when I've been lucky to get some video support, I stopped expecting it to properly handle external connections to projectors.
Fast forward to 2005 and Excalibur, my Dell Precision 670 workstation. This system is the synthesis of 11 years of hateful experiences with video cards. The nvidia Quadro NVS 280 card which shipped with the machine lived in the PCI Express slot but was advertised a "2D accelerated card!" For a long time the nvidia drivers didn't recognize it, and when they did, the OpenGL drivers hated it. When they finally did start to work, performance was terrible.
I replaced the NVS card with an nvidia 6600 GT PCIE card. It was reasonably fast, but had the annoying habit of causing the system to freeze intermittantly when doing OpenGL work. The freeze in question was a hard freeze - one had to power off to regain control of the system. The card worked 100% flawlessly under Windows. Performance between Linux and Windows was noticebly different. Whereas the card produced expected performance under Windows, it was about 50% slower under Linux. Yes, I made sure I wasn't using Mesa drivers. I also made sure I removed the X screen saver, because it would randomly make the system lock up.
So, eventually, I replaced the 6600GT PCIE card with an ATI X300 card. This card was labelled specifically as supported by Linux. In fact, it has been the most stable of all of the cards I've used. It worked (generally) out of the box. Of course, this was with software rendering using mesa. I just spent the past 1 1/2 hours getting the fglrx modules installed and working. There is still a little bit of a bug with the AIGLX extensions from the livna package. I'm running a plain vanilla FC6 system (I haven't even tried to compile a custom kernel).
Shouldn't this stuff just work? Why is it that Linux, after 12 years, has yet to find a stable video driver framework? If Linux is going to be successful as a desktop replacement, this needs to be fixed and fixed now. Why? Because Microsoft & Apple have it right. My dad can buy a PC, plug it in, and it just works. I'm probably going to switch to Apple, and I expect that it will just work.
Open Source Just Doesn't Work
Now, you may argue that Linux does work, its the video card driver vendors. Those evil companies that don't want their cards to work with Linux. They keep their drivers closed source so no one can see their code. Lets get those evildoers and smoke 'em out. Closed source drivers are irrelevant. I've spent the last 12 years trying to make Linux work. For a long time I was an evangelist for the cause. But one cannot make a convincing argument for a technology that is stuck chasing the tail lights of its competition. Windows is totally closed source, and yet things work. MacOS is closed source, and yet things work. Solaris is closed source, and yet things work.
Linux will probably continue to gain ground in the server room (although frankly I prefer Solaris). But I do not see any significant market penetration until I can hand my Dad a Linux DVD and tell him go put this in your drive, and all will be well.