VMWare ESXi Passthrough and Freenas

I recently had user colicstew comment that they could not get Freenas 9.10.2 U3 to see drives on his LSI2116.  Luckily I have some unused 2TB Drives and LSI 2116 Controller as part of my Xeon D Lab.

IMG_0561.jpgIMG_0560.jpg

First step was to passthrough my LSI2116 Controller and Reboot the Server.  You should see Enabled/Active for the LSI Controller which we do

Snip20170511_1.png

Next I created a VM. Snip20170511_2.png

I chose to install the OS on one of my VMware Datastores.

Snip20170511_3.png

On the Hardware Settings I set as shown.

Snip20170511_4.png

Then I clicked Add other device -> PCI Device and Add the LSI ControllerSnip20170511_5.png

Finally I added a CD-ROM Drive and attached the Freenas ISO downloaded from here.

Snip20170511_7.png

Snip20170511_8.png

Finally we boot the VM

Snip20170511_9.png

Then we select Install at the first Screen

Snip20170511_13.png

One of the oddities I ran into while creating this tutorial was the key sequence to release the mouse from the VMWare Remote Console was the escape key in the Freenas installer that bounces you back to Initial Setup Screen, so the remaining pictures are via my phone.

On the next screen you will be warned about only having 4GB of memory.  Since I’m only using this as a test system, I am not concerned.  If you were running this as an actual file server you would want at a minimum 1GB of Memory per TB of storage.

IMG_0562.jpgOn the this screen we see five drives.  One is our drive we will install FreeNAS OS to, and the other four are 2TB Hitachi Drives I attached to the Xeon D Board.

IMG_0563.jpg

Next you are warned that FreeNAS will clear the drive you are installing to.

IMG_0564.jpg

Next you are asked to set the root password.

IMG_0565.jpg

Finally you are asked to choose a BIOS Mode.  

IMG_0566.jpg

At this point, I went to install the system, but no matter what I did, it would not install the OS, failing with the below error.

Snip20170511_15.png

The problem here is that the drive that we are installing the OS to is thin provisioned.  FreeNAS does not like this.

Snip20170511_16.png

Fix is to create the disk as “Thick provisioned, eagerly zeroed”.

Snip20170511_17.png

Once that was fixed, the OS installation continued without issue.

Snip20170511_18.png

Once completed, you will see the following success message.Snip20170511_19.png

Then, I chose option four to shutdown the system.  Be sure to remove the DVD Drive, or the system will continue to boot from the Installer.  Once that is completed your system will work as expected.

Snip20170511_23.png

Unfortunately, I had no issues getting the system up and running.  colicstew feel free to email me via the contact method and we can see about troubleshooting this issue.  I would start with verifying you have the correct breakout cable.  I used one of the four breakout cables from the Motherboard.

 

Advertisements

NAS OS Install – A Journey in Patience

capture

I’m in the process of setting up the OS for the new NAS server, and I have to say, the OpenIndiana installer is slow.  I waited 10 minutes to after seeing the Sun Copyright banner, and then 15, then 20, and eventually 30 minutes.  At that point I decided to go download Freenas 9.10.1 and prepared to mount it via the IPMI on the SuperMicro Xeon D board.  But low and behold we have movement! Then we stalled again.  I was starting to have flashbacks of Solaris on Sun Hardware with Software RAID on 16 Drive Storage arrays running early version of ZFS.

Then I got smart, or so I thought.   I took the USB Expander and removed it.  Then I plugged the USB directly into the motherboard.  It didn’t do a thing unfortunately.

capture

So instead, I decided to a mount an ISO via the IPMI and go with FreeNas. So everything was going well.

capture

Okay we are making progress

capture

After logging in and doing some basic setup, I created the first Volume.

capture

Speed tests were as expected, one mirror does one disk performance, 2 mirrors do two disk performance.

I still plan on installing OpenIndiana on this machine in the future and benchmarking it as well.  My 10G Switch and new Card has arrived, so I’ll be sure to put those benchmarks up as well.

Upcoming NAS Server

dsc_0002

I’ve finally outgrown my Synology 416J NAS.  It’s a well built device, but you are completely limited as to expanding or improving the performance.  Currently an 8TB Seagate USB3 Device has the same or better CIFS speed than my NAS.

I plan on thoroughly benchmarking my current setup in SHR, RAID10 and RAID5.  Currently I have 2X3TB and 2X6TB in the Synology NAS and in both cases I’m maxing out the Gigabit Ethernet.

I’ve been investigating using FreeNas/OmniOS/Napp-It as a storage backend for Plex, Windows Backups and general file storage.

Here is a tease of the hardware

4X HGST 6TB He Drives

64GB DDR4 ECC

SuperMicro X10SDV-4C-TLN4F Motherboard

LSI 9211-8i HBA

Supermicro SSD-DM016-PHI 16GB SATA DOM (Disk on Module)

Fractal Node 304 Case

OCZ RD400/400A M.2 SSD

Intel X520-DA2 10G Network Card for Direct Connect to Plex VM

I’m still debating about adding another 12TB of HGST 6TB Drives.   I’m also trying to figure out a workflow of how to benchmark the current NAS Device as I’d like to show myself that what I’ve built is better than what I have. Also for the ZFS, I’m unsure of my ZIL/SLOG/L2ARC configuration.  I’ll take it as a learning experience and see what would work best for performance.