Short note, hope this might save someone else a few "bang head here" headaches:
need: cheap VM environment with the ability to run 64-bit VM's in VMWare ESXi4.1
Solution: Dell 6850 w/ 4x Intel Xeon 7140m processors and 32GB RAM (cost, $1220)
Problem number 1: 6850's require 200-240V power. Since I was going to use this in my home, I don't have a circuit with that voltage (think electric stove or electric dryer; these are the plugs with a diameter about half that of a CD). had I noticed this before the purchase, wouldn't have purchased the system. However, I was able to use it for work.
replaced existing CPUs and hooked up power, only to run into...
problem 2: system wouldn't post, wouldn't get into the BIOS config, and reported that the processors were incompatible with the system. BIOS was at latest for 6850's (A06), motherboard part is WC983, Rev A00. Double checked the 6850 documentation PDF that Dell put out and confirmed that the 7140M is indeed compatible (read the fine print, it was used for a bench mark).
"No problem, I'll just call Dell." Make sure you have your system ownership information updated before doing so, or you'll get nowhere. That was problem number 3.
After talking with Dell over two days (seems even they have to dig for this info) it turns out that you need the following parts for the Xeon 7140M (SL9HA) processors (these are mandatory):
2x Dell Part YC902 (Voltage Regulator Modules)
4x Dell Part WG189 (Heatsinks for Motherboard, N6164 will not work)
1x Dell Part PD838 (3rd VRM for Cache) CANNOT BE PART K5331!
1x Dell Part RD318 (6850 Mother Board)
4x Dell Part ND891 (Memory Risers, part N4867 did not work)
1x The rest of the server
So, is the 6850 a cheap, viable option for running 64-bit VM's in ESXi4.1? Viable, with a few caveats. First, make sure you have the correct voltage! Second, you must have the Xeon 7000 series processors (Intel part number SL9HA, SL9HB, SL9HC, SL8UD, SL8UB) since these are the only ones with the VT-x technology you need. Third, make sure you have the voltage regulators to support the processors. Finally, make sure you have the right 6850 Motherboard (RD318 if you want to run the SL9HA's).
As for cheap? well, after getting the rest of the parts needed (VRMs, Rails, disk), the total for the unit I've put together will be about $1220 for 8 cores @ 3.4 GHz, 64MB cache, 24GB RAIDed RAM, 800 FSB, and 2x 36GB 10K U320 SCSI HDD in RAID 1. Stallard, Inc sells comparable 1950s for about $3460 each (w/o RAM RAID), but they have to be Gen III and still only capable of 24MB cache max. You might be able to find everything on ebay for a bit cheaper, but it's still going to cost you more than $1000. I should probably also mention that this system is not hosting the storage (using a 2850 running OpenFiler for that for the time being, 6x 146GB 10K U320 SCSI = 730GB usable in 5+1 RAID 5, approx $400).
For those of you looking to repeat what I've done, here's a list:
Dell 6850 (liquid8technology.com has them w/ 16GB RAM for a good price on ebay)
4x 4GB (2x2GB) PC2-3200 DDR2 RAM (server-ram on ebay)
2x 36GB 10K U320 SCSI HDD (check your back plane, could be SAS)
4x Intel Xeon 7140m (SL9HA)
4x Heatsinks (WG189)
1x 6850 RD318 Motherboard
4x ND891 Memory Risers
2x YC902 Voltage Regulator Modules
1x PD838 Voltage Regulator Module (hard to find on Ebay, can be as much as $150 elsewhere)
Rails, of course