I decided to go with OmniOS as it seems to be the OS everyone recommends for ZFS and NappIt and it works with the 10G networking. Installation went smooth and without issues
Unfortunately, that was the only thing that went well. No configuration would allow me to get a 10G write above 180-190MB/s which is just not acceptable. I would have expected a greater than 60% increase in speed.
Then it dawned on me. I’m being limited by the speed of the media I’m using to copy my files from (a USB 3.0 Drive). In theory a SATA Interfaced drive is only going to hit 160-190MB/s.
So then I copied a file locally and witnessed the same speeds, and copied that file from a Local Drive on a VM to the mounted SMB share, and boom there was my speeds, topping out at 400+MB/s
Nerd Blogging - Tech Blogging 