Execute the command (replace the IP address with the IP address of your target server):
On Linux, to determine the device name for your source disk, use the following command:
On OpenSolaris or Solaris, to determine the device name for your source disk, use the following command:
Thanks to the folks over at genunix.org, I have a freshly burned CD-ROM of OpenSolaris 2010 preview build 134. I manage a Sun 7410 Unified Storage Appliance (SAN) at work and I’m very interested in the inner workings of the COMSTAR framework used by this appliance. Unfortunately, using the command prompt on the Sun 7410 (without having Sun Support on the phone) voids the support contract. Luckily, the Amber Road GUI is feature rich and I haven’t found anything I need the command prompt for other than tinkering.
To learn more about COMSTAR my plan is to install OpenSolaris on my personal machine and configure it as an iSCSI target backed by a ZFS file system. If the experiment goes well, we may decide to use similar configurations to expand our SAN at work.
I am especially interested to test an iSCSI target backed by a ZFS volume running deduplication and find out what kind of ratio I can get. I’m afraid that the files will not line up on block boundaries and there might be poor deduplication performance… lets find out if I’m wrong.
Lets talk requirements for this experiment:
Hardware List:
Step 1: Install OpenSolaris
Use the GUI, have the partitioner use an entire disk for the installation (installing ZFS on a slice huts performance because the drive’s cache is not used effectively). Currently, OpenSolaris 2010.03 build 133 does not have a text installer so you will need to use the graphical installer.
Step 2: Configure Mirroring
Prepare the 2nd disk to be used for the root mirror using the format command. Delete any existing partitions, create a new Solaris2 partition using the entire disk. For my 320GB SATA disk the partition looks like this when it’s ready.
Partition Status Type Start End Length %
========= ====== ============ ===== === ====== ===
1 Solaris2 1 38912 38912 100
Step 3: Copy the Volume Table of Contents from the disk used by the GUI Installer to the disk you just partitioned.
admin@opensolaris:/etc# prtvtoc /dev/rdsk/c6d0s2 | fmthard -s – /dev/rdsk/c7d0s2
fmthard: New volume table of contents now in place.
Step 4: Attach the 2nd disk to the root mirror
admin@opensolaris:/etc# zpool attach -f rpool c6d0s0 c7d0s0
Please be sure to invoke installgrub(1M) to make ‘c7d0s0′ bootable.
Make sure to wait until resilver is done before rebooting.
Step 5: Install Grub on the disk you just added to the root mirror
admin@opensolaris:/etc# installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c7d0s0
Step 6: You’re basically done with root, but…
Make sure the new root mirror finishes resilvering before you try to reboot. In the case of my 320GB disk resilvering took 10 minutes and I was able to install grub while the resilvering was running.
admin@opensolaris:/etc# zpool status
pool: rpool
state: ONLINE
scrub: resilver completed after 0h10m with 0 errors on Mon Feb 22 21:42:48 2010
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c6d0s0 ONLINE 0 0 0
c7d0s0 ONLINE 0 0 0 7.68G resilvered
errors: No known data errors
Step 7: Configure Networking
Use the GUI application located at System –> Administration –> Network to manually configure networking. Once the GUI settings are complete, execute the following commands to use DNS for Internet name resolution. I have two NICS, i will use DHCP on one for Internet access and static on the other for iSCSI access.
iSCSI Interface properties
Internet Interface properties
admin@opensolaris:~# cd /etc
admin@opensolaris:/etc# cp nsswitch.conf nsswitch.conf.original
admin@opensolaris:/etc# cp nsswitch.dns nsswitch.conf
admin@opensolaris:/etc# pkg install storage-server
Step 8: Enable the STMF Service
admin@opensolaris:/etc# svcadm enable stmf
Step 9: Create a new ZFS volume
replace the device names with the names of your disks from Step 2 above. I am using a 64k block size to optimize VMware performance.
admin@opensolaris:/etc# zpool create -f pool1 c6d1 c7d1
admin@opensolaris:/etc# zfs create -b 64k -V 885G pool1/vol1
admin@opensolaris:/etc# zfs set compression=on pool1/vol1
Note: you probably want compression unless your CPU is old and you have a ton of disks. ZFS uses LZJB compression by default and most likely your CPU has more compression bandwidth than your disks have platter bandwidth. Multiple benchmarks are available online. Here is one that is well written and uses ZFS and compression for database storage.
Step 10: Create an iSCSI Logical Unit using the volume.
admin@opensolaris:/etc# sbdadm create-lu /dev/zvol/rdsk/pool1/vol1
Created the following LU:
GUID DATA SIZE SOURCE
——————————– ——————- —————-
600144f01db0830000004b83463c0001 950261514240 /dev/zvol/rdsk/pool1/vol1
Step 11: Make the Logical Unit available to all hosts
substitute the GUID from the previous step
admin@opensolaris:/etc# stmfadm add-view 600144F01DB0830000004B83463C0001
Step 12: Enable an iSCSI Target Port
admin@opensolaris:/etc# svcadm enable -r svc:/network/iscsi/target:default
svcadm: svc:/milestone/network depends on svc:/network/physical, which has multiple instances.
Step 13: Configure the iSCSI target for discovery
admin@opensolaris:/etc# devfsadm -i iscsi
That’s it for the OpenSolaris Side. In my next post, we’ll configure the Windows iSCSI initiator and test deduplication & compression performance for NTFS on iSCSI. If you’re really hardcore and want maximum performance, you can disable the OpenSolaris GUI with the following command.
admin@opensolaris:/etc# svcadm disable gdm
Thanks to DarkStar for guidance on how to configure a root mirror with a GUI installer, this is not well documented.