[2014-06-29 this blog post has been updated to reflect installation on a 64GiB Mac Pro (not a 16GiB Mac Mini, which didn't have enough RAM to deploy Cloud Foundry)]
In this blog post, we describe the procedure to deploy VMware ESXi and VMware vCenter on an Apple Mac Mini running VMware Fusion.
This 64GiB Mac Pro is the World’s Smallest Installation of Cloud Foundry. The Bottle of Pellegrino is for scale. Not pictured: 4TB External USB 3 Drive.
Mac Pro Configuration
We went with the following configuration:
- 3.7GHz quad-core with 10MB of L3 cache (i.e. Intel Xeon E5-1620 v2)
- D500 Graphics Card .
- 512MB Flash (note: we regretted this decision; we wished we had opted for the $500-more-expensive 1TB)
- 64GiB  RAM
- External 4TB USB 3 Drive
Why the Mac Pro? It’s the only machine that Apple sells that accepts more than 32GB of RAM.
1. Network Settings
We allocate the following subnet for our CF installation:
- Subnet: 10.9.8.0/24
- Subnet Mask: 255.255.255.0
- Default Route: 10.9.8.1
- DNS: 10.9.8.1 (yes, in our case the gateway is also the nameserver)
We add these DNS Entries for our key hosts:
- 10.9.8.1 gateway.cf.nono.com
- 10.9.8.10 esxi.cf.nono.com
- 10.9.8.20 vcenter.cf.nono.com
- 10.9.8.30 opsmgr.cf.nono.com
- 10.9.8.40 *.cf.nono.com (App Domain)
We set this range to be our reserved IPs:
Which means this range is our available IPs:
We appreciate that it’s not always easy to allocate a /24 subnet, that in certain organizations allocation of more than a few IP addresses requires flexing of interdepartmental muscle. We are contemplating a future blog post describing a set-up requiring but a few IP address.
2. Prevent Mission Control from Hijacking F11 and F12
Go to System Preferences → Keyboard → Shortcuts → Mission Control:
- uncheck Show Desktop F11
- uncheck Show Dashboard F12
3. A Windows Machine
For a brief portion of the install we will need a Windows machine (in order to deploy the vCenter .ova file to the ESXi host).
4. Download VMware Software
- Download ESXi: We browse to VMware (you may need to create a VMware account). The path we follow to download is My VMware → Downloads → All Downloads → VMware vSphere → VMware ESXi 5.5.0 Update 1. The download should be approximately 335 MB.
- Download vCenter Server: Again, we browse to VMware and follow this path: My VMware → Downloads → All Downloads → VMware vSphere → VMware vCenter Server 5.5 Update 1a Appliance. The download should be approximately 2GB.
We download this file to our Windows machine because we will use the vSphere client on our Windows machine to install vCenter on our ESXi.
Make sure that the downloaded file has a .ova extension and not a .ovf extension. Certain browsers (i.e. Chrome) append the wrong extension to the downloaded file.
4. Download Cloud Foundry Software
- Download Pivotal CF (you’ll need to create an account and agree to the EULA). Click Download. The download should be approximately 5.3GB
1. Configure ESXi 5.5 VM Settings
- Bring Up VMware Fusion
- ⌘N (File → New)
- Select Install from disc or image; click Continue
- Click Use another disc or disc image…
- Browse to your ESXi ISO image (e.g. VMware-ESXi-5.5U1-Rollup_2ISO.iso) and click Open
- Click Continue
- Click Customize Settings
- Change the name to CF ESXi and click Save (choose a location on the 4TB External Drive, e.g. our location is “/Volumes/Big Disk/vmware/”) (note: if you decide to place the Virtual Machine on the Mac Pro’s flash drive, remember to exclude that location from Time Machine backups)
- Adjust the following Settings:
- Processor & Memory:
- 4 Processor Cores
- 49152 MB RAM
- click Show All
- Network Adapter
- Select Bridged Networking → Ethernet (our Mac Pro is using its ethernet port, not its WiFi)
- Hard Disk
- 750.00 GB
- click Apply; click Show All
- Close the Settings window
2. Install ESXi:
- Click the ▶ button to start the ESXi Virtual Machine.
- You’ll see an ESXi-5.5U1-1623589-RollupISO-standard Boot Menu panel. It has a ten-second timeout. Don’t do anything; let it time out.
- You will see a warning, “A virtual machine is attempting to monitor…”. Enter your password; click OK
- You’ll see a panel that says Welcome to the VMware ESXi 5.5.0 Installations; Press Enter to continue
- press F11 to Accept and Continue
- Choose the 750 GiB drive; press Enter
- press Enter (US Default)
- Enter your password twice and hit Enter
- Press F11 to confirm install
- Press Enter to reboot after the message ESXi 5.5.0 has been successfully installed.
3. Configure ESXi Networking
- Wait until ESXi finishes rebooting
- Press F2 twice (yes, twice) to login
- Enter your username and password, then press Enter
- Select Configure Management Network; press Enter
- Select IP Configure; press Enter
- Select Set static IP address…
- IP Address: 10.9.8.10
- Subnet Mask: 255.255.255.0
- Default Gateway: 10.9.8.1
- Press Enter
- Select DNS Configuration
- Select Use the following DNS server…
- Primary DNS Server: 22.214.171.124 (unless you have a local DNS server you’d like to use)
- Hostname: esxi.cf.nono.com
- Press Enter
- Press Y (Apply changes and restart management network)
- Select Test Management Network
- Press Enter; every test should return OK
- Press Enter; Press Esc to log out
4. Configure ESXi License, NTP
Note: configuring NTP is optional; we just can’t help ourselves—NTP is the crack cocaine of system administrators.
We do the following on the Windows machine.
- Browse to the ESXi host. Your browser will present you with a warning that the SSL certificate is unverified; ignore the warning.
- Click Download vSphere Client
- Open the downloaded client; Click Yes to install
- If prompted, click Yes again; click Next, accept the terms, click Next, click Install, click Finish
- Double-click the VMware vSphere Client on the desktop
- IP address / Name: esxi.cf.nono.com
- User name: root
- Password: ESXi root password
- click Login
- When warned about the certificate, do the following:
- check Install this certificate…
- click Ignore
- We see a warning that “Your evaluation license will expire in 60 days!”. Click on OK
- Click the Configuration tab
- Software → Licensed Features
- ESX Server License Type: click Edit…
- Select Assign a new license key to this host
- Click Enter Key…
- New license key: enter-the-license-key-purchased-from-VMware
- click OK
- Configure NTP
- Click the Configuration tab
- Software → Time Configuration → Properties
- check NTP Client Enabled; click Options…
- click General, click Start and stop with host
- click NTP Settings, check Restart NTP service…, click Add…, type time.apple.com, click OK
- click OK; click OK
1. Initial Install
- We do the following on the Windows vSphere client:
File → Deploy OVF Template…
- Click Browse…
- Browse to the downloaded vCenter .ova file (e.g. VMware-vCenter-Server-Appliance-126.96.36.19900-1750781_OVF10.ova); select it and click Open
- Click Next (OVF Template Details)
- Click Next (default name of “VMware vCenter Server Appliance”)
- Click Next (Thick Provision)
- Check Power on after deployment; Click Finish
2. Root Password, Network Configuration
We do the following on the Windows machine, in the VMware vSphere Client.
- In the left-hand navbar, click the “+” next to esxi.cf.nono.com to expand the inventory list of VMs
- In the left-hand navbar, select VMware vCenter Server Appliance, Make sure it’s powered on (click Power on the virtual machine
- Click Console tab
- Click inside the Console tab (your mouse pointer will disappear)
- When Prompted with “NO NETWORKING DETECTED”, press Enter
- change password:
New password: some-new-password
(you can ignore the warning, “BAD PASSWORD: is too simple”)
Retype new password: some-new-password
In this section, we configure the networking for the vCenter server. This is vital.
We also change the hostname from “localhost.localdom” to “vcenter.cf.nono.com” for æsthetic reasons. Also, we enable IPv6, but once again our motives are æsthetic rather than functional—you may dispense with those steps.
y (IPv6) (choose n if you don’t have IPv6 or aren’t sure)
n (IPv6 no DHCP)
Enter (IPv6 address)
64 (IPv6 prefix)
y (IPv6 correct)
n (no DHCP)
2 (default gateway)
press Enter (IPv6)
188.8.131.52 (DNS server 2)
0 (review changes)
ping -c 2 google.com # check net settings
We reboot the vCenter VM because we are superstitious; perhaps this step is not necessary:
shutdown -r now
1. Press Ctrl-Alt to liberate your mouse pointer
3. First Time vCenter Set-Up
The following can be done on any machine; it does not need to be done from the Windows machine.
- Browse to the vCenter client, port 5480 (in our example, https://vcenter.cf.nono.com:5480 (if you see a No data received or The connection was reset message, you’re probably using http instead of https)
User name: root
check Accept license agreement
Select Configure with default settings; click Next
The next portion is important, at least it will be important 90 days from now when the root password expires and we’re locked out of our vCenter. To avoid lock-out, we do the following:
- Click on the Admin tab
- Administrator password expires: Select No
- Click Submit
4. vCenter License, Datacenter, and Cluster
We browse to our vCenter (no special port, not port 5480), e.g. https://vcenter.cf.nono.com. We confirm [with our browser] that our SSL cert is unverified, that our connection is untrusted.
Click Log in to vSphere Web Client. You may have to re-confirm [with our browser] that our SSL cert is unverified.
Log in with user name root and the password we set earlier.
We see a yellow band on the top of the page with the words, “There are vCenter Server systems with expiring license keys…”.
- Click Details…
- Click Add new license keys to vSphere
- Click + to add a new license key
- Enter the new license key that we’ve purchased from VMware, e.g.
- Click Finish
- Click on the vCenter Server Systems tab toward the top of the page
- Click Assign License Key…
- Select the key we previously typed in; click OK
- Click the × on the yellow band at the top of the page to dismiss
- Click the home icon (top of the page, towards the left) to return to the main screen
- Click the vCenter icon (we can choose the icon from the navbar on the left or the icon in the middle of the screen—they both take us to the same place)
- Click Datacenters on the lefthand side navbar
- Click the icon to add a datacenter (buildings with a green “+”). A New Datacenter window will pop up.
- (we are naming our datacenter with the unimaginative default name, “Datacenter”). Select the vCenter and Click OK
- Click on the Create a New Cluster icon (several computers with a green “+”). A New Cluster window will pop up:
- Name: Cluster
- Checked: DRS
- click OK
Add ESXi Server
Click the Add a Host icon (a computer with a green “+”). An Add Host window will pop up.
- Name and Location
- Host name or IP address: esxi.cf.nono.com
- click Next
- Connection settings
- User name: root
- Password: whatever-we-set-the-password-to
- Click Yes to verify the authenticity of the host
- (Host summary) click Next
- (Assign license) click Next
- (Lockdown mode) click Next
- (VM location) click Next
- (Ready to complete) click Finish
Congratulations, we have created an IaaS. Ready for more? Let’s install Cloud Foundry’s Ops Manager and deploy BOSH in the subsequent post, World’s Smallest IaaS, Part 2.
Some of the ESXi and vCenter configuration was drawn from internal Cloud Foundry documents.
1 Note: purchasing the D500 over the less-expensive D300 has nothing to do with Cloud Foundry; anyone purchasing a Mac Pro to run Cloud Foundry should opt for the D300 Graphics Card, which is currently $400 less expensive than the D500. The decision to purchase a D500 was related to gaming, which is not an appropriate topic for a blog post, even though the D500 is quite adequate to play ESO at 1920×1200 with ultra-high settings, easily delivering over 30fps (frames per second).
2 We didn’t purchase Apple RAM; we purchased Crucial RAM. Apple charges $1,300 for 64GB (over the base option of 12GB). We purchased 2 x 32GB kits, which consists of two sticks apiece, for a grand total of 4 x 16GB sticks, at a cost of (after tax & shipping) $822.12.
Do not make the mistake that we did, thinking we could mix the Crucial RAM with the Apple RAM: Originally we had purchased only 32GiB from Crucial, with the belief that we could retain 8GiB of the 12GiB that were included with our Mac Pro, for a grand total of 40GiB. We were doomed to disappointment. The Crucial RAM was RDIMM; the Apple RAM was UDIMM. “Do not mix UDIMMs and RDIMMs,” says Apple on its Mac Pro memory specification page. If you mix them, like we did, the Mac Pro will not boot but instead will beep plaintively.