1 ZFS Encryption
Test Plan
PSARC/2007/261
2 Introduction
2.1
Authors
|
Name
|
Email
|
Extension /
Region
|
|
Grace Tang
|
|
|
2.2
Test Sponsor
|
Name
|
Email
|
Extension /
Region
|
|
Rob Sohigian
Jim Walker
|
|
|
2.3
Test Plan Approvals
|
Title
|
Name
|
Version
|
Date
|
|
Development
Lead
|
Darren Moffat
|
0.3
|
2007-12-10
|
|
Development
Manager
|
Steven Detar
|
0.3
|
2007-12-17
|
|
Test Sponsor
|
Jim Walker, Robert Sohigian
|
1.1
|
2008-4-18
|
2.4
External Open Source Test Software
2.5
Revision History
|
Date
(yyyy-mm-dd)
|
Revision
|
Comments
|
Approval/Status
|
|
2007-11-05
|
0.1
|
Initial draft
for I-team review
|
submitted to
I-team
|
|
2007-11-23
|
0.2
|
Incorporate
I-team comments
|
submitted to
I-team
|
|
2007-12-10
|
0.3
|
Incorporate
I-team comments
|
approved by
I-team
|
|
2008-1-9
|
0.4
|
Incorporate
test sponsors' comments
|
not approved
|
| 2008-1-30 |
1.0
|
Incorporate
test sponsors' comments |
approved by test sponsors
|
| 2008-4-18 |
1.1
|
Update Zones, TX and ZFS boot testing
|
approved by test sponsors
|
2.6
References
2.7
Staffing
|
Name
|
Role
|
% Commitment
|
Duration
|
|
Grace Tang
|
QE Engineer
|
80%
|
5 months
|
|
ZFS
Encryption I-team
|
contribution
|
|
|
2.8
Glossary
|
Term
|
Definition
|
|
aes-128-ccm, aes-256-ccm
|
two kinds of ZFS encryption policy. aes stands
for AES
encryption algorithm. 128 or 256 are key length in bits used for the
encryption algorithm. ccm stands for CCM encryption mode.
|
|
Cryptographic
Framework
|
Sun
Cryptographic Framework provides cryptographic services to users and
applications through commands, a user-level programming interface, a
kernel programming interface, and user-level and kernel-level
frameworks. ZFS Encryption is a consumer of the Cryptographic Framework
uses the kernel and user-level programming interfaces.
|
|
ZFS
|
a file system originally created by Sun for
Solaris. The features of ZFS include high storage capacity, integration
of the concepts of file system and volume management, a novel on-disk
structure, lightweight instances, and easy storage pool management.
|
|
ZFS dataset
|
a generic
name for the following ZFS entities: clones, file systems, snapshots or
volume
|
3 Test Program
Summary
ZFS Encryption
will provide on disk encryption/decryption support for ZFS datasets.
It will cover the addition of encryption and decryption to the ZFS IO
pipeline and the key management for ZFS datasets.
This test
program will qualify:
-
data in a ZFS
dataset will be encrypted on disk with the encryption property set to
on, aes-128-ccm and aes-256-ccm.
-
checksum will be forced to sha256+mac with
encryption set to on/aes-128-ccm/aes-256-ccm.
-
the command
line interface changes of zfs/zpool.
Test assertions
will be developed to test encryption, new checksum type and
restriction and CLIs. Once assertions have been fully documented they
will be reviewed by the I-team for coverage and correctness and
modified or extended as needed. New automated functional tests will
be developed per assertions. The new tests will be put back into
STC2.0 gate.
To ensure no regression is introduced by ZFS
encryption, the ZFS test suite should be run. There will be two kinds
of regression testing. One is with encryption set to off, the other
is on. Because the encryption property can only be set at dataset
creation time and can not be later changed, the ZFS test suite can't
be reused directly as regression tests when encryption is set to on.
ZFS testing team agreed to add a wrapper on the test suite to allow
encryption. If the wrapper wouldn't work, it's possible to pick some
tests in the suite and modify setup scripts, then run these tests as
regression tests with encryption.
Performance
testing will be run periodically.
4 Operational
Factors
4.1
Assumptions
Assumption
#1: Test machines
have at least two scratch disks will be available to run the ZFS test
suite.
Assumption
#2: A T2 test
machine will be available to test Niagara 2 cryptographic accelerator
works for AES and SHA operations.
Assumption
#3: This test
plan assumes the ZFS Encryption test team can get help resource from
Perf-PIT and the ZFS test team.
4.2
Dependencies
Dependency
#1:
ZFS
Encryption has a hard dependency on RFE 6631490 (ZIL refactor for
zfs crypto). This project can't be putback without ZIL crypto.
4.3
Risks
Risk
#1:
-
Description:
This test plan bases on the ZFS
Encryption design document. There will be design changes as the
project development goes on and there are some
unresolved
issues. Design shifts will affect the test plan and development.
-
Likelihood of Occurrence:
High
-
Mitigation
and Contingency Plan: Tests will be developed first for stable
function and CLIs.
5 Test
Development Information
5.1
Test Development
|
Test Name
|
Automated or
Manual
|
Test Type
|
Project
Component
|
Description
|
|
fs_encrypt
|
Automated
|
Functional
|
encryption
|
New tests to
verify ZFS file systems data and metadata will be encrypted with the
encryption property set to on, aes-128-ccm or aes-256-ccm at creation
time. The correct key must be presented to access encrypted data and
metadata. The encryption property is inherited by default and can be
obviously overriden.
|
|
zvol_encrypt
|
Automated
|
Functional
|
encryption
|
New tests to
verify ZFS volumes will be encrypted with the encryption property set
to on, aes-128-ccm or aes-256-ccm at creation time. The correct key
must be presented to access encrypted volumes.
|
|
snapshot_encrypt
|
Automated
|
Functional
|
encryption
|
New tests to
verify snapshots of encrypted file systems/volumes will be encrypted.
|
|
clone_encrypt
|
Automated
|
Functional
|
encryption
|
New tests to
verify clones of encrypted file systems/volumes will be encrypted. The
encryption property is inherited by default and can be obviously
overriden.
|
|
import_encrypt
|
Automated
|
Functional
|
encryption
|
New tests to verify the importing behaviours
of a pool. Importing a pool with a key, the unencrypted and encrypted
datasets are accessible. Importing an encrypted
pool without a key, verify the unencrypted datasets are accessible,
while the encrypted ones are not.
|
|
zil_encrypt
|
Automated
|
Functional
|
encryption
|
New tests to
verify when tuning the ZIL flush time, encrypted datasets are always
encrypted on disk. Default and separate log devices will be tested.
|
|
l2arc_encrypt
|
Automated
|
Functional
|
encryption
|
New tests to
verify when adding/removing L2ARC devices to/from a pool, encrypted
datasets are always encrypted on disk.
|
|
sha256+mac_checksum
|
Automated
|
Functional
|
checksum
|
New tests to verify when encryption is set
to aes-128-ccm and aes-256-ccm, checksum will be sha256+mac and the
user couldn't change the checksum type.
|
|
zpool_create
|
Automated
|
Functional
|
cli
|
New tests to
verify changes of 'zpool create' work per its man page.
|
|
zpool_set_kek
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zpool set kek' works per its man page.
|
|
zpool_key
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zpool key' works per its man page.
|
|
zpool_get_keystatus
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zpool get keystatus' works per its man page.
|
|
zpool_status
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zpool status' works per its man page.
|
|
zfs_create
|
Automated
|
Functional
|
cli
|
New tests to
verify changes of 'zfs create' work per its man page.
|
|
zfs_set_kek
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zfs set kek' works per its man page.
|
|
zfs_key
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zfs key' works per its man page.
|
|
zfs_get_keystatus
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zfs get keystatus' works per its man page.
|
|
zfs_status
|
Automated
|
Functional
|
cli
|
New tests to
verify 'zfs status' works per its man page.
|
|
delegation
|
Automated
|
Functional
|
delegation
|
New tests to
verify delegations introduced by ZFS Encryption work.
|
|
cross_arch
|
Automated
|
Functional
|
cross_arch
|
New tests to
verify encrypted datasets can be moved across architecture without data corruption.
|
6 Areas of
Testing
6.1
Functional Testing
The following
tests will be run to verify the functionality introduced by ZFS
Encryption:
-
encryption
-
checksum
-
cli
-
delegation
-
cross_arch
6.2
Regression Testing
The following
tests will be run to make sure no regression introduced by ZFS
Encryption.
6.3
Conformance Testing
None.
6.4
Stress/Robustness Testing
The following ZFS
stress tests which are running by the ZFS project will be performed
with the encryption property set to on and off. The ZFS test team
will help to do stress testing.
-
zfs_stress002, zfs_stress003, zfs_stress004,
zfs_stress005, zfs_stress006 - zfs stress load tests
6.5
Performance Testing
To see the impact
introduced by ZFS Encryption, all ZFS performance tests will be run
with the encryption property set to on and off. The ZFS Encryption
test team will provide instructions to the Ireland Perf-PIT team to
run the following benchmarks:
6.6
Memory Leak Testing
Mdb:findleaks
will be used to find out memory leaks in kernel caused by the changes
made to kernel by the project. This will be accomplished by first
running functional tests followed by "reboot -d" to take
the crash dump and performing ::findleaks on the core.
6.7
Required Solaris Feature Testing
Functional testing
will be done in global and local zones to verify per dataset key
management as well creation of encrypted datasets are allowed in global
and local zones, while per pool key operations are only allowed in
global zones.
1) ZFS Encryption will be tested using the TX test suite.
2)
Functional testing will be done in the Trusted path and labelled zones
to verify per dataset key management as well creation of encrypted
datasets are allowed in the Trusted path and labelled zones, while per
pool key operations are only allowed in the Trusted path.
ZFS Encryption is
to add a new feature of ZFS. Changes will be made to the ZFS test
suite to set the encryption property to on and off to verify no
regression.
Functional
testing will be done in a Solaris guest domain via xVM on x86 or
LDOMs on SPARC to verify ZFS Encryption is not affected by or affect
xVM and LDOMs.
6.8
Interoperability Testing
None.
6.9
Testing Not Covered
The project will
do some changes to the Solaris Cryptographic Framework, this test
program will not cover that part.
6.10
Documentation Testing
The following
project man pages will be verified for correctness as part of the
test program.
-
zfs changes
-
zpool changes
6.11
Internationalization Testing
Internationalization
testing will be done to verify the output of new zpool/zfs command
lines can be localized correctly.
7 Test Execution
7.1
Hardware Test Configurations
The following HW
platforms will be tested:
Niagara 2
Hardware crypto accelerator will be tested to perform cryptographic
operations, so the SPARC machine will be Sun SPARC Enterprise T5220.
To run the ZFS test suite, all test machines should have at least
two scratch disks.
7.2
Software Test Configurations
The following
Solaris kernel modes will be tested:
-
32 debug
-
32 non-debug
-
64 debug
-
64 non-debug
7.3
Extrapolation Strategy (if applicable)
None.
7.4
Pre-Integration Run
-
Is a
pre-integration PIT run required? (REQUIRED, RECOMMENDED, NONE)? There
is no required PIT run. Since BSTE has resources to run the ZFS test
suite on x86 platform, we will get those results from them.
-
If a
pre-integration PIT run is required or recommended, should the run be
the OS PIT, Net PIT or both? OS PIT
7.5
Test Execution Matrix
The tests
summarized in table 7.5.2 will be run on the configurations listed in
table 7.5.1.
Table 7.5.1:
Machine HW/SW Configurations
|
#
|
Type
|
Name
|
Architecture
|
CPU/Memory
|
Kernel
|
Crypto
Accelera
|
|
1
|
Sun
SPARC Enterprise T5220
|
purple-197.east
|
sun4v
|
64*1417 MHz/16256MB
|
64
non-debug
|
Niagara
2
|
|
2
|
Sun
SPARC Enterprise T5220
|
TBD
|
sun4v
|
TBD
|
64
debug
|
Niagara
2
|
|
3
|
Sun
Fire V40z
|
stinger4p-3.west
|
amd64
|
4*1992
MHz/2048MB
|
64
non-debug
|
none
|
|
4
|
Sun
Fire V40z
|
stinger4p-2.west
|
amd64
|
4*1992 MHz/8192MB
|
64
debug
|
none
|
|
5
|
Sun
Fire V60x
|
TBD
|
x86
|
TBD
|
32
non-debug
|
none
|
|
6
|
Sun
Fire V60x
|
TBD
|
x86
|
TBD
|
32
debug
|
none
|
|
6
|
Sun
Fire X4150
|
TBD
|
Intel64
|
TBD
|
64
non-debug
|
none
|
|
7
|
Sun
Fire X4150
|
TBD
|
Intel64
|
TBD
|
64
debug
|
none
|
Table 7.5.2: Test
Suite Execution
|
Test Type
|
Test Suite
|
Architecture
|
|
SPARC debug
|
SPARC
non-debug
|
amd64 debug
|
amd64
non-debug
|
x86
debug
|
x86
non-debug
|
Intel64
debug
|
Intel64
non-debug
|
|
functional
|
encryption
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
functional
|
checksum
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
functional
|
cli
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
X
|
|
regression
|
ZFS test
suite without/with encryption
|
X
|
|
X
|
|
X
|
|
X
|
|
|
regression
|
manual ZFS boot tests
|
X
|
|
X
|
|
X
|
|
X
|
|
|
stress
|
ZFS stress
tests without/with encryption
|
X
|
|
X
|
|
X
|
|
X
|
|
|
performance
|
netbench,
specweb99, oltp-net, sfs, kenbus, bigdir2, bonnie, iozone3, postmark,
filebench
|
|
X
|
|
X
|
|
X
|
|
X
|
|
memory leak
|
memory leak
|
X
|
|
X
|
|
X
|
|
X
|
|
|
zone
|
zone
|
|
|
|
|
|
|
|
|
|
Trusted
Extensions
|
TX
|
|
|
|
|
|
|
|
|
|
xVM
|
xVM
|
|
|
X
|
|
X
|
|
X
|
|
|
LDOMs
|
LDOMs
|
X
|
|
|
|
|
|
|
|
|
internationalization
|
internationalization
|
X
|
|
X
|
|
X
|
|
X
|
|
|
documentation
|
documentation
|
|
|
|
|
|
|
|
|
8
Schedules and Milestones
9
Post-Integration Testing Information
9.1
Test Suite Integration Requirements
|
Test/Test
Suite Name
|
Source
Integration Location
|
Target
Integration Date
|
Comments
|
|
zfs-crypto
|
STC 2.0
|
TBD
|
Tests are,
referencing 5.1, encryption, checksum, cli, delegation and cross_arch tests. All the tests will
be under /ws/onnv-stc2/src/suites/security/zfs-crypto.
|
9.2
Test Suite Execution Integration Requirements
|
Test/Test
Suite Name
|
Execution
Group
|
Comments
|
|
zfs-crypto
|
ONPIT
|
After the
test suite is put back into STC 2.0 gate, it will be integrated into
ONPIT.
|
9.3
Other Post-Integration Requirements
None.
10 Solaris
Update Release (if applicable)
N/A
Copyright 2008 Sun Microsystems, Inc. All rights reserved.