To maintain a sane development environment, we pull fresh copies of the PDFs every month from production. We pull from production to staging, then from production to development. The PDFs are stored on two separate servers that both run NTFS. We use Microsoft SyncToy to sync the PDFs across environments. The process can take several hours for each environment. The network load is high due to the PDFs being stored on multiple servers.
I recently had an idea. What if we store the PDFs on our ZFS NAS? We could use ZFS snapshotting and rsync to refresh the environments. We can do that on a regular basis via a cron job. ZFS snapshots take a few seconds and rsync is a really efficient tool. No network traffic will be involved since the synchronization is taking place all on the same server.
Here are the commands we would run:
DATE=`date '+%F_%T'` zfs snapshot tank/site_data/prod/PDFs@$DATE rsync -a /tank/site_data/prod/PDFs/.zfs/snapshot/$DATE/ /tank/site_data/dev/PDFs/
I really like this solution. Right now, we have to jump through a lot of hoops to sync up these PDFs. This will save us time, space, and internal bandwidth.
This article originally posted on my tech blog.