OpenSolaris

You are not signed in. Sign in or register.

Dynamic LUN expansion and Thin Provisioning support in COMSTAR

Thin Provisioning means that if you want to create a LUN of say 100 Gigabyte in size, you dont have to right away allocate 100 Gigabytes from you backend storage pool for this LUN. In fact you dont have to allocate any space at all to begin with. You just define the size during initialization of the LUN so that the target mode software knows what to tell the initiators when they ask about the size of the LUN. But the actual space for the LUN's backend storage is only allocated when the initiator(s) write any data to that space.

Dynamic LUN expansion allows you to not even bother about defining the size you will need over the life of the LUN. You can just define what you need today (or in near future) and then grow the size of the LUN as the demand for the space goes up. This is also useful in the case where you dont want to create very large Thin provisioned LUNs in the beginning.

COMSTAR supports both of these features as shown in the examples below.

Creating a thin provisioned SCSI logical unit

Create a zero byte file. Initialize it to be 1Terabyte in size and register it with the framework. Also expose it to all the initiators.

# touch /lun0 
# sbdadm create-lu -s 1T /lun0
 
Created the following LU:
 
              GUID                    DATA SIZE           SOURCE
------------------------------—  -----------------—  --------------—
6000ae40c5000000000047a22c1c0001      1099511562240    /lun0
# stmfadm add-view 6000ae40c5000000000047a22c1c0001
#

But the size on the disk is …

# ls -l /lun0
-rw-r—r—   1 root     root        8192 Jan 31 13:14 /lun0
#

The 8K is due to some metadata which gets written to the file as a part of initialization.

Now lets initialize it from the initiator side. In case of solaris, this will be like running format and labeling the disk. After thats done, lets look at the file size again.

# ls -l /lun0
-rw-r—r—   1 root     root     1099506052608 Jan 31 13:17 /lun0
#

…So it took 1 terabytes of space ? Not really. Lets look at the real space used.

# ls -s /lun0
 208 /lun0
#

So it only took 208 disk blocks or in other words 104 Kilobytes. So this is basically a file with a "hole". Most filesystems (ufs, zfs etc.) support files with holes in them. In reality what happened was that the initiator wrote to the beginning of the LUN and then it lseek(2) to the end and wrote a backup label there. The remaining space in the middle which was never written did not use any disk blocks on the back end storage. But the backend usage will grow as the initiator will write more and more data in the middle.

Couple of things to note are:

  • The maximum size supported by COMSTAR for thin provisioned LUNs is 8 Exabytes (8192 Petabytes). This size gets further limited by what is supported by underlying filesystem on which the backend storage pool resides. e.g. UFS will not support more than 2 Terabytes of LUNs. But with zfs you will be able to go all the way upto 8 Exabytes.

  • If you dont use Thin provisioning, the LUN size will be derived from the size of the file or raw device at the time of LU initialization.
  • Some applications on the initiator side will try to write to the entire disk as a part of their initialization sequence for the disk. This initiator functionality will force the target side to allocate all the space from the backend storage in the beginning itself. To prevent this from happening "Dynamic LUN Expansion" as described below should be used.

Growing the LUN dynamically

sbd supports modify-lu option to modify the size of a LU dynamically. Lets try changing the size of the LU we just created.

# sbdadm modify-lu -s 1500G /lun0
LU modified Successfully.

Lets look at the size again.

# sbdadm list-lu
 
Found 1 LU(s)
 
              GUID                    DATA SIZE           SOURCE
------------------------------—  -----------------—  --------------—
6000ae40c5000000000047a22c1c0001      1610612670464    /lun0
#

When the size of the LU is modified, the initiators accessing this LU get notified via a 'SCSI check condition' that 'CAPACITY HAS CHANGED'. Of course the initiators and the filesystems on the initiators must be capable of handling this notification and adjust the size accordingly.