summaryrefslogtreecommitdiffhomepage
path: root/zfs.html.markdown
diff options
context:
space:
mode:
Diffstat (limited to 'zfs.html.markdown')
-rw-r--r--zfs.html.markdown400
1 files changed, 400 insertions, 0 deletions
diff --git a/zfs.html.markdown b/zfs.html.markdown
new file mode 100644
index 00000000..74487e35
--- /dev/null
+++ b/zfs.html.markdown
@@ -0,0 +1,400 @@
+---
+category: tool
+tool: zfs
+contributors:
+ - ["sarlalian", "http://github.com/sarlalian"]
+filename: LearnZfs.txt
+---
+
+
+[ZFS](http://open-zfs.org/wiki/Main_Page)
+is a rethinking of the storage stack, combining traditional file systems as well as volume
+managers into one cohesive tool. ZFS has some specific teminology that sets it appart from
+more traditional storage systems, however it has a great set of features with a focus on
+usability for systems administrators.
+
+
+## ZFS Concepts
+
+### Virtual Devices
+
+A VDEV is similar to a raid device presented by a RAID card, there are several different
+types of VDEV's that offer various advantages, including redundancy and speed. In general
+VDEV's offer better reliability and safety than a RAID card. It is discouraged to use a
+RAID setup with ZFS, as ZFS expects to directly manage the underlying disks.
+
+Types of VDEV's
+* stripe (a single disk, no redundancy)
+* mirror (n-way mirrors supported)
+* raidz
+ * raidz1 (1-disk parity, similar to RAID 5)
+ * raidz2 (2-disk parity, similar to RAID 6)
+ * raidz3 (3-disk parity, no RAID analog)
+* disk
+* file (not recommended for production due to another filesystem adding unnecessary layering)
+
+Your data is striped across all the VDEV's present in your Storage Pool, so more VDEV's will
+increase your IOPS.
+
+### Storage Pools
+
+ZFS uses Storage Pools as an abstraction over the lower level storage provider (VDEV), allow
+you to separate the user visable file system from the physcal layout.
+
+### ZFS Dataset
+
+ZFS datasets are analagous to traditional filesystems but with many more features. They
+provide many of ZFS's advantages. Datasets support [Copy on Write](https://en.wikipedia.org/wiki/Copy-on-write)
+snapshots, quota's, compression and deduplication.
+
+
+### Limits
+
+One directory may contain up to 2^48 files, up to 16 exabytes each. A single storage pool
+can contain up to 256 zettabytes (2^78) of space, and can be striped across 2^64 devices. A
+single host can have 2^64 storage pools. The limits are huge.
+
+
+## Commands
+
+### Storage Pools
+
+Actions:
+* List
+* Status
+* Destroy
+* Get/Set properties
+
+List zpools
+
+```bash
+# Create a raidz zpool
+$ zpool create bucket raidz1 gpt/zfs0 gpt/zfs1 gpt/zfs2
+
+# List ZPools
+$ zpool list
+NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
+zroot 141G 106G 35.2G - 43% 75% 1.00x ONLINE -
+
+# List detailed information about a specific zpool
+$ zpool list -v zroot
+NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
+zroot 141G 106G 35.2G - 43% 75% 1.00x ONLINE -
+ gptid/c92a5ccf-a5bb-11e4-a77d-001b2172c655 141G 106G 35.2G - 43% 75%
+```
+
+Status of zpools
+
+```bash
+# Get status information about zpools
+$ zpool status
+ pool: zroot
+ state: ONLINE
+ scan: scrub repaired 0 in 2h51m with 0 errors on Thu Oct 1 07:08:31 2015
+config:
+
+ NAME STATE READ WRITE CKSUM
+ zroot ONLINE 0 0 0
+ gptid/c92a5ccf-a5bb-11e4-a77d-001b2172c655 ONLINE 0 0 0
+
+errors: No known data errors
+
+# Scrubbing a zpool to correct any errors
+$ zpool scrub zroot
+$ zpool status -v zroot
+ pool: zroot
+ state: ONLINE
+ scan: scrub in progress since Thu Oct 15 16:59:14 2015
+ 39.1M scanned out of 106G at 1.45M/s, 20h47m to go
+ 0 repaired, 0.04% done
+config:
+
+ NAME STATE READ WRITE CKSUM
+ zroot ONLINE 0 0 0
+ gptid/c92a5ccf-a5bb-11e4-a77d-001b2172c655 ONLINE 0 0 0
+
+errors: No known data errors
+```
+
+Properties of zpools
+
+```bash
+
+# Getting properties from the pool properties can be user set or system provided.
+$ zpool get all zroot
+NAME PROPERTY VALUE SOURCE
+zroot size 141G -
+zroot capacity 75% -
+zroot altroot - default
+zroot health ONLINE -
+...
+
+# Setting a zpool property
+$ zpool set comment="Storage of mah stuff" zroot
+$ zpool get comment
+NAME PROPERTY VALUE SOURCE
+tank comment - default
+zroot comment Storage of mah stuff local
+```
+
+Remove zpool
+
+```bash
+$ zpool destroy test
+```
+
+
+### Datasets
+
+Actions:
+* Create
+* List
+* Rename
+* Delete
+* Get/Set properties
+
+Create datasets
+
+```bash
+# Create dataset
+$ zfs create tank/root/data
+$ mount | grep data
+tank/root/data on /data (zfs, local, nfsv4acls)
+
+# Create child dataset
+$ zfs create tank/root/data/stuff
+$ mount | grep data
+tank/root/data on /data (zfs, local, nfsv4acls)
+tank/root/data/stuff on /data/stuff (zfs, local, nfsv4acls)
+
+
+# Create Volume
+$ zfs create -V zroot/win_vm
+$ zfs list zroot/win_vm
+NAME USED AVAIL REFER MOUNTPOINT
+tank/win_vm 4.13G 17.9G 64K -
+```
+
+List datasets
+
+```bash
+# List all datasets
+$ zfs list
+NAME USED AVAIL REFER MOUNTPOINT
+zroot 106G 30.8G 144K none
+zroot/ROOT 18.5G 30.8G 144K none
+zroot/ROOT/10.1 8K 30.8G 9.63G /
+zroot/ROOT/default 18.5G 30.8G 11.2G /
+zroot/backup 5.23G 30.8G 144K none
+zroot/home 288K 30.8G 144K none
+...
+
+# List a specific dataset
+$ zfs list zroot/home
+NAME USED AVAIL REFER MOUNTPOINT
+zroot/home 288K 30.8G 144K none
+
+# List snapshots
+$ zfs list -t snapshot
+zroot@daily-2015-10-15 0 - 144K -
+zroot/ROOT@daily-2015-10-15 0 - 144K -
+zroot/ROOT/default@daily-2015-10-15 0 - 24.2G -
+zroot/tmp@daily-2015-10-15 124K - 708M -
+zroot/usr@daily-2015-10-15 0 - 144K -
+zroot/home@daily-2015-10-15 0 - 11.9G -
+zroot/var@daily-2015-10-15 704K - 1.42G -
+zroot/var/log@daily-2015-10-15 192K - 828K -
+zroot/var/tmp@daily-2015-10-15 0 - 152K -
+```
+
+Rename datasets
+
+```bash
+$ zfs rename tank/root/home tank/root/old_home
+$ zfs rename tank/root/new_home tank/root/home
+```
+
+Delete dataset
+
+```bash
+# Datasets cannot be deleted if they have any snapshots
+zfs destroy tank/root/home
+```
+
+Get / set properties of a dataset
+
+```bash
+# Get all properties
+$ zfs get all zroot/usr/home │157 # Create Volume
+NAME PROPERTY VALUE SOURCE │158 $ zfs create -V zroot/win_vm
+zroot/home type filesystem - │159 $ zfs list zroot/win_vm
+zroot/home creation Mon Oct 20 14:44 2014 - │160 NAME USED AVAIL REFER MOUNTPOINT
+zroot/home used 11.9G - │161 tank/win_vm 4.13G 17.9G 64K -
+zroot/home available 94.1G - │162 ```
+zroot/home referenced 11.9G - │163
+zroot/home mounted yes -
+...
+
+# Get property from dataset
+$ zfs get compression zroot/usr/home
+NAME PROPERTY VALUE SOURCE
+zroot/home compression off default
+
+# Set property on dataset
+$ zfs set compression=gzip-9 mypool/lamb
+
+# Get a set of properties from all datasets
+$ zfs list -o name,quota,reservation
+NAME QUOTA RESERV
+zroot none none
+zroot/ROOT none none
+zroot/ROOT/default none none
+zroot/tmp none none
+zroot/usr none none
+zroot/home none none
+zroot/var none none
+...
+```
+
+
+### Snapshots
+
+ZFS snapshots are one of the things about zfs that are a really big deal
+
+* The space they take up is equal to the difference in data between the filesystem and its snapshot
+* Creation time is only seconds
+* Recovery is as fast as you can write data.
+* They are easy to automate.
+
+Actions:
+* Create
+* Delete
+* Rename
+* Access snapshots
+* Send / Receive
+* Clone
+
+
+Create snapshots
+
+```bash
+# Create a snapshot of a single dataset
+zfs snapshot tank/home/sarlalian@now
+
+# Create a snapshot of a dataset and its children
+$ zfs snapshot -r tank/home@now
+$ zfs list -t snapshot
+NAME USED AVAIL REFER MOUNTPOINT
+tank/home@now 0 - 26K -
+tank/home/sarlalian@now 0 - 259M -
+tank/home/alice@now 0 - 156M -
+tank/home/bob@now 0 - 156M -
+...
+
+Destroy snapshots
+
+```bash
+# How to destroy a snapshot
+$ zfs destroy tank/home/sarlalian@now
+
+# Delete a snapshot on a parent dataset and its children
+$ zfs destroy -r tank/home/sarlalian@now
+
+```
+
+Renaming Snapshots
+
+```bash
+# Rename a snapshot
+$ zfs rename tank/home/sarlalian@now tank/home/sarlalian@today
+$ zfs rename tank/home/sarlalian@now today
+
+# zfs rename -r tank/home@now @yesterday
+```
+
+Accessing snapshots
+
+```bash
+# CD Into a snapshot directory
+$ cd /home/.zfs/snapshot/
+```
+
+Sending and Receiving
+
+```bash
+# Backup a snapshot to a file
+$ zfs send tank/home/sarlalian@now | gzip > backup_file.gz
+
+# Send a snapshot to another dataset
+$ zfs send tank/home/sarlalian@now | zfs recv backups/home/sarlalian
+
+# Send a snapshot to a remote host
+$ zfs send tank/home/sarlalian@now | ssh root@backup_server 'zfs recv tank/home/sarlalian'
+
+# Send full dataset with snapshos to new host
+$ zfs send -v -R tank/home@now | ssh root@backup_server 'zfs recv tank/home'
+```
+
+Cloneing Snapshots
+
+```bash
+# Clone a snapshot
+$ zfs clone tank/home/sarlalian@now tank/home/sarlalian_new
+
+# Promoting the clone so it is no longer dependent on the snapshot
+$ zfs promote tank/home/sarlalian_new
+```
+
+### Putting it all together
+
+This following a script utilizing FreeBSD, jails and ZFS to automate
+provisioning a clean copy of a mysql staging database from a live replication
+slave.
+
+```bash
+#!/bin/sh
+
+echo "==== Stopping the staging database server ===="
+jail -r staging
+
+echo "==== Cleaning up existing staging server and snapshot ===="
+zfs destroy -r zroot/jails/staging
+zfs destroy zroot/jails/slave@staging
+
+echo "==== Quiescing the slave database ===="
+echo "FLUSH TABLES WITH READ LOCK;" | /usr/local/bin/mysql -u root -pmyrootpassword -h slave
+
+echo "==== Snapshotting the slave db filesystem as zroot/jails/slave@staging ===="
+zfs snapshot zroot/jails/slave@staging
+
+echo "==== Starting the slave database server ===="
+jail -c slave
+
+echo "==== Cloning the slave snapshot to the staging server ===="
+zfs clone zroot/jails/slave@staging zroot/jails/staging
+
+echo "==== Installing the staging mysql config ===="
+mv /jails/staging/usr/local/etc/my.cnf /jails/staging/usr/local/etc/my.cnf.slave
+cp /jails/staging/usr/local/etc/my.cnf.staging /jails/staging/usr/local/etc/my.cnf
+
+echo "==== Setting up the staging rc.conf file ===="
+mv /jails/staging/etc/rc.conf.local /jails/staging/etc/rc.conf.slave
+mv /jails/staging/etc/rc.conf.staging /jails/staging/etc/rc.conf.local
+
+echo "==== Starting the staging db server ===="
+jail -c staging
+
+echo "==== Make sthe staging database not pull from the master ===="
+echo "STOP SLAVE;" | /usr/local/bin/mysql -u root -pmyrootpassword -h staging
+echo "RESET SLAVE;" | /usr/local/bin/mysql -u root -pmyrootpassword -h staging
+```
+
+
+### Additional Reading
+
+* [BSDNow's Crash Course on ZFS](http://www.bsdnow.tv/tutorials/zfs)
+* [FreeBSD Handbook on ZFS](https://wiki.freebsd.org/ZF://wiki.freebsd.org/ZFS)
+* [BSDNow's Crash Course on ZFS](http://www.bsdnow.tv/tutorials/zfs)
+* [Oracle's Tuning Guide](http://www.oracle.com/technetwork/articles/servers-storage-admin/sto-recommended-zfs-settings-1951715.html)
+* [OpenZFS Tuning Guide](http://open-zfs.org/wiki/Performance_tuning)
+* [FreeBSD ZFS Tuning Guide](https://wiki.freebsd.org/ZFSTuningGuide)