• Backups vs. Archiving
• Methodological Considerations
• Local Solutions
• Online Solutions

• Hardware failure
  • Disk drive in the computer
  • Solid state memory devices (internal drives and external thumb drives)
• Unexpected power problem
  • Power surge/drop
  • Something touches your computer too much (child/pet)
• Catastrophic event
  • Hurricane
  • other
• Device accidents
  • Stolen
  • Dropped/spilled/etc
  • Pressed format accidentally

• Data Protection
  • Includes topics such as backups, archives, and preservation
  • also includes physical security, encryption, and others not addressed here (for later)...
• Terms “backups” and “archives” are often used interchangeably, but do have different meanings
  • Backups: a copy (or copies) of the original file is made before the original is overwritten
  • Archives: preservation of the file
• Data Preservation
  • Includes archiving in addition to processes such as data rescue, data reformatting, data conversion, metadata

• Backups
  • Used to take periodic snapshots of data in case the current version is destroyed or lost
  • Backups are copies of files stored for short or near-long-term
  • Often performed on a somewhat frequent schedule
• Archives
  • Used to preserve data for historical reference or potentially during disasters
  • Archives are usually the final version, stored for long-term, and generally not copied over
  • Often performed at the end of a project or during major milestones
  
  It is a good idea to have multiple copies of your backups and archives, in case one copy fails.

• Who is responsible for backup ?
• How often do you backup ?
• Partial vs. full backups ?
• Non-digital backups ?
• Where (literally) will the backups be located ?
• Do the backups need a description (metadata) ?
• Manual vs automatic ?
• Recovery procedures ?
• Verification – how do you know the backup was successful ?
• How long do you keep your backups ?
• What happens when the project ends ?

• Data conversions and formats
• Versioning
• File Naming

• A design firm was handling their own backups. The system was working fine and the backup software was reporting that the data was successfully backed up.

• The administrator checked the backups immediately after they were done and confirmed they were good.

• Always check file sizes after backing up
  • Or at least check periodically
• The MD5 checksum
  • With this you can monitor the integrity of your data over time
  • Other checksums are CSC and SHA
  • Like a fingerprint for a file
  • Command line: md5sum
  
  
  
  

• Cloud based services (box, dropbox, etc.) are based on Folder Synchronization
  • Only copies newer files (that have changed or been created)
  • Thus contains a "mirror image"
• There are commercial and free folder synchronization tools
  • All have privacy and user data issues
  • All based on “command line” tools that already exist on your computer
• Command line interface (CLI) synching
  • Mac or Linux: rsync
  • Windows: xcopy or robocopy
  
  
  
  

• Local Machines
  • Hard disk in computer
  • External hard drives
• Online Solutions
  • Networked drives (personal cloud)
  • Repositories
  • Versioning Tolls
  • The "cloud"

• Pros
  • High level of control over file system, naming, and physical location of disk
  • Easy to backup
  • Convenient
• Cons
  • Risk of malware (virus)
  • Risk of theft, damage, loss, etc
  • System can eventually corrupt the disk (especially pcs)
  • Finite lifespan

• Pros
  • High level of control over file system, naming, and physical location of disk
  • Likely has backup and maintenance schedule
  • Possible duplicate (mirror) images – RAID systems
  • Safe physical location
  Redundant Array of Independent Disks
• Cons
  • Expensive to maintain
  • Migration can be difficult
  • Susceptible to catastrophic events

• Pros
  • Drives are cheap (sort of) and portable
  • Convenient
  • Memory is cheap and portable
• Cons
  • Connection technologies change (USB, Firewire, SATA, and so on)
  • Drive failure (both spinning drives and memory devices)
  • Easily damaged, stolen or lost
  • Finite space for large projects multiple drives may be necessary
  • Malware can be propagated (think unsafe sex)

• Pros
  • Massive amunts of data, cheap
  • Fast backup
  • Reusable
• Cons
  • Slow retrevial
  • Degradation over time
  • Installation and maintenance is expensive

• Pros
  • Drives are cheap (sort of) and portable
  • Convenient access from anywhere
  • Easy to install and sync
  • Private: password protected
• Cons
  • Upload/download bottlenecks
  • Susceptible to acts catastrophic events
  • Needs permanent power
  • Needs IP address

• Pros
  • Maintained by others
  • Your data is accessible, visible
  • Mostly cost free
• Cons
  • Like journals, can be well recognized, but not necessarily
  • Takes time to format data and metadata correctly
  • Your data is accessible, visible

• For programmers only?
  • Github: http://github.com
  • Bitbucket: http://bitbucket.org
  • Subversion: https://subversion.apache.org
• Project Management (for fee service)
  • Basecamp: http://basecamp.com
  • Teamwork: http://teamwork.com
  
  
  
  

• Pros
  • Maintained by others
  • Often open text-based formats
  • Excellent version control and backup
• Cons
  • Steep learning curve
  • Doesn’t handle proprietary data well
  • Your data is accessible, visible (sometimes)

• Pros
  • No failure or backup worries (they do it)
  • Can be secure (depends)
  • Convenient
  • Good for catastrophic events
• Cons
  • Upload/download bottlenecks
  • Fees?
  • Long-term? No standards?
  • How to get copies of all your data (try this for google drive)
  • No control, AND you are responsible if something is hacked (according to US government export laws)

• Google Drive? Drop Box? Sky Drive? iCloud?
  • Drop Box is good for temporary sharing
  • Google Drive is good for collaborative work: synchronous file editing with multiple users
  • What about security?? privacy??
• BOX - https://www.box.com/
  • University of Miami has an affiliation
  • All content is encrypted
  • All platforms are supported including smart phones
  • Recently became “unlimited”
  
  http://www.miami.edu/it/index.php/about_it/aas/ps/documentation/box/
  
  
  

If you need more space at the University of Miami go to the Center for Computational Science (CCS)


• They sell space using the condo model
• ~28 TB for $7,000 (2015) – in perpetuity
• Not necessarily backed up (but it is raided)
• Talk to Joel Zysman at advanced computing resources

• Mac and linux: rsync
• PC: xcopy or robocopy

• Mac: Applications/Utilities/Terminal
• PC: Start Button->search “cmd”

• PC: xcopy

C:\> xcopy <source> <destination> [<options>]

C:\> xcopy c:\Users\tnorris\Documents\MapData\*.* G:\MapData /D /S /Y

This copies all NEWER files from the MapData directory on the local machine
to the backup MapData folder on an external hard drive.

C:\> xcopy /?

This will show all of the options for the xcopy command. You can see that:
/D tells xcopy to only copy newer files
/S goes through all sub-directories
/Y tells xcopy to proceed without asking the user to confirm (be careful!!)
            

• Mac or linux: rsync

% rsync [<options>] <source> <destination>

% rsync -arvu /Users/tnorris/Documents/MapData/* /Volumes/MyDrive/MapDataFB
 
This copies all NEWER files from the MapData directory on the local machine to 
the backup MapDataFB folder on an external hard drive named “MyDrive”.

% man rsync

This will show all of the options for the rsync command. You can see that 
 -a tells rsync preserve archival information (date stamps, owners, permissions)
 -r tells rsync to go through all sub-directories
 -v tells rsync to tell you what it is doing (which files it copied)
 –u will only update newer files and skip older files.
            

• Security and Privacy
• File System Organization
• Midterm Assignment

• Pros
  • High level of control over file system, naming, and physical location of disk
  • Easy to backup
  • Convenient
• Cons
  • Risk of malware (virus)
  • Risk of theft, damage, loss, etc
  • System can eventually corrupt the disk (especially pcs)
  • Finite lifespan

• Pros
  • High level of control over file system, naming, and physical location of disk
  • Likely has backup and maintenance schedule
  • Possible duplicate (mirror) images – RAID systems
  • Safe physical location
  Redundant Array of Independent Disks
• Cons
  • Expensive to maintain
  • Migration can be difficult
  • Susceptible to catastrophic events

• Pros
  • Drives are cheap (sort of) and portable
  • Convenient
  • Memory is cheap and portable
• Cons
  • Connection technologies change (USB, Firewire, SATA, and so on)
  • Drive failure (both spinning drives and memory devices)
  • Easily damaged, stolen or lost
  • Finite space for large projects multiple drives may be necessary
  • Malware can be propagated (think unsafe sex)

• Pros
  • Drives are cheap (sort of) and portable
  • Convenient access from anywhere
  • Easy to install and sync
  • Private: password protected
• Cons
  • Upload/download bottlenecks
  • Susceptible to acts catastrophic events
  • Needs permanent power
  • Needs IP address

• Pros
  • No failure or backup worries (they do it)
  • Can be secure (depends)
  • Convenient
  • Good for catastrophic events
• Cons
  • Upload/download bottlenecks
  • Fees?
  • Long-term? No standards?
  • How to get copies of all your data (try this for google drive)
  • No control

• Must meet federal compliance requirements for security and privacy
• As examples: HIPAA, FERPA, FISMA

• Name, SSN, Date of Birth, Drivers License, Address, IP Address, Phone Number, anything that can uniquely identify a person
• NIH Guide for Identifying Sensitive Information
  https://oma.od.nih.gov/DMS/Documents/Privacy/Guide%20for%20Handling%20Sensitive%20Information%20at%20NIH.pdf

• Use automatic updates (mostly for virus issues)
• Use anti-virus software (also anti-intrusion)
• Use a firewall (also password protect file-sharing)
• Never connect to untrusted wireless connections
• Computer disposal?
• Know HTTPS, SSH/SCP, sFTP
• Understand certificate errors
• Do not send confidential email
• Public computers
  • Always log out
  • Never leave data/files

• Set screen to lock
• This is not foolproof, any hacker knows how to get past a password. See: http://pogostick.net/~pnh/ntpasswd/ for example.

• UM Policy: http://www.miami.edu/it/index.php/policies/
• Mac: System Preferences: Security and Privacy: FileVault
• PC: Control Panel: BitLocker Drive Encryption

• Mac: Finder->Preferences: Advanced Tab, “Empty Trash Securely”
• PC: Eraser (open source GPL): http://eraser.heidi.ie/

$ openssl

$ openssl des3 -in test.txt -out encrypted.txt

$ openssl des3 -d -in encrypted.txt -out testout.txt
            

• your username as a password (even backwards or mixed up).
• using any name, or any word in any language.
• obvious personal information (your year of birth, phone number, national insurance number, address, etc.).
• all digits, or just one letter.
• real words with only one or two obvious digit substitutions, like 'p4ssword' or '5ecret'.
• fewer than eight characters ("brute force" attack cracks 7 letters in a few minutes).
• characters from books, films, etc. (Gandalf, Sherlock), band names, song titles etc. (no matter how obscure).
• passwords that are too easy or too difficult to type

1. You lost control of your data because of the fine print in a user agreement.
   
   Do a google search “ shady user agreement”
2. You sent out a public link to a Google Doc so others could view and edit.
   
   Invite people through emails
3. Your cloud account gets hacked (bad password).
   
   Better password management
4. You use the same password for every app on your phone.
   
   See above
5. Web trackers are storing information on the sites you visit online.
   
   Private browsing? Don’t stay logged in to google all the time?
6. You granted an application (smartphone) every permission under the sun.
   
   Be thoughtful when you install and run new apps
7. A small mobile app startup you know nothing about has access to your banking data.
   
   Let you bank manage your banking data