Automated gmail backup via IMAP

This is how I setup automated gmail backup using IMAP via mbsync. Parts are MacOS X specific.

  • Enable IMAP in your gmail account.
  • Install mbsync, if using MacPorts do: sudo port install isync
  • Create a new directory ~/Backup/gmail
  • Save this certificate as ~/Backup/gmail/gmail.pem. This is the gmail IMAP SSL certificate, retrieved via openssl s_client -connect imap.gmail.com:993 -showcerts

    -----BEGIN CERTIFICATE-----
    MIIDYzCCAsygAwIBAgIQcdBJTwL0s64EVDDexAG1jTANBgkqhkiG9w0BAQUFADCB
    zjELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJ
    Q2FwZSBUb3duMR0wGwYDVQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UE
    CxMfQ2VydGlmaWNhdGlvbiBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhh
    d3RlIFByZW1pdW0gU2VydmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNl
    cnZlckB0aGF3dGUuY29tMB4XDTA3MDUxMTE1NTUzMFoXDTA4MDUxMDE1NTUzMFow
    aDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCkNhbGlmb3JuaWExFjAUBgNVBAcTDU1v
    dW50YWluIFZpZXcxEzARBgNVBAoTCkdvb2dsZSBJbmMxFzAVBgNVBAMTDmltYXAu
    Z21haWwuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDp8NJaYAUMfsA4
    uW1y5wXh6sE31Uh/s0dLgeTu4NbzA36Ru3bmpB4zxCNUgnNT73OfhhtTitx8BPoB
    zdY4Tgwc1asVXSw0h2iNoj6/KIiCv4r5FFqWMQxdHZh3n6/VJnNzCPnD62fJy9D8
    j9jIfU6NGM91+zgsEexU7JuHB+y2jwIDAQABo4GmMIGjMB0GA1UdJQQWMBQGCCsG
    AQUFBwMBBggrBgEFBQcDAjBABgNVHR8EOTA3MDWgM6Axhi9odHRwOi8vY3JsLnRo
    YXd0ZS5jb20vVGhhd3RlUHJlbWl1bVNlcnZlckNBLmNybDAyBggrBgEFBQcBAQQm
    MCQwIgYIKwYBBQUHMAGGFmh0dHA6Ly9vY3NwLnRoYXd0ZS5jb20wDAYDVR0TAQH/
    BAIwADANBgkqhkiG9w0BAQUFAAOBgQBIuR0Dr4wURb1CjxMVjWA9/lPZl2f2Kr++
    naPcrIw+gJMLwU88OCfs7XqOHQ/n9dRnQ+mXcrmJKHVQAh0d038JKOaglVBn6LdX
    nJovtY8DyeYPXMGHdIwxPj7H583HQRGqkDF1usr29X3JZxcpPi3ICk+lRYoSHBvH
    /MXIPo3WJA==
    -----END CERTIFICATE-----
    
  • Save this certificate as ~/Backup/gmail/thawte.pem. This is the CA for the gmail certificate, downloaded from http://www.thawte.com/roots/.

    -----BEGIN CERTIFICATE-----
    MIIDJzCCApCgAwIBAgIBATANBgkqhkiG9w0BAQQFADCBzjELMAkGA1UEBhMCWkEx
    FTATBgNVBAgTDFdlc3Rlcm4gQ2FwZTESMBAGA1UEBxMJQ2FwZSBUb3duMR0wGwYD
    VQQKExRUaGF3dGUgQ29uc3VsdGluZyBjYzEoMCYGA1UECxMfQ2VydGlmaWNhdGlv
    biBTZXJ2aWNlcyBEaXZpc2lvbjEhMB8GA1UEAxMYVGhhd3RlIFByZW1pdW0gU2Vy
    dmVyIENBMSgwJgYJKoZIhvcNAQkBFhlwcmVtaXVtLXNlcnZlckB0aGF3dGUuY29t
    MB4XDTk2MDgwMTAwMDAwMFoXDTIwMTIzMTIzNTk1OVowgc4xCzAJBgNVBAYTAlpB
    MRUwEwYDVQQIEwxXZXN0ZXJuIENhcGUxEjAQBgNVBAcTCUNhcGUgVG93bjEdMBsG
    A1UEChMUVGhhd3RlIENvbnN1bHRpbmcgY2MxKDAmBgNVBAsTH0NlcnRpZmljYXRp
    b24gU2VydmljZXMgRGl2aXNpb24xITAfBgNVBAMTGFRoYXd0ZSBQcmVtaXVtIFNl
    cnZlciBDQTEoMCYGCSqGSIb3DQEJARYZcHJlbWl1bS1zZXJ2ZXJAdGhhd3RlLmNv
    bTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA0jY2aovXwlue2oFBYo847kkE
    VdbQ7xwblRZH7xhINTpS9CtqBo87L+pW46+GjZ4X9560ZXUCTe/LCaIhUdib0GfQ
    ug2SBhRz1JPLlyoAnFxODLz6FVL88kRu2hFKbgifLy3j+ao6hnO2RlNYyIkFvYMR
    uHM/qgeN9EJN50CdHDcCAwEAAaMTMBEwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG
    9w0BAQQFAAOBgQAmSCwWwlj66BZ0DKqqX1Q/8tfJeGBeXm43YyJ3Nn6yF8Q0ufUI
    hfzJATj/Tb7yFkJD57taRvvBxhEf8UqwKEbJw8RCfbz6q1lu1bdRiBHjpIUZa4JM
    pAwSremkrj/xw0llmozFyD4lt5SZu5IycQfwhl7tUCemDaYj+bvLpgcUQg==
    -----END CERTIFICATE-----
    
  • Save this config file in ~/Backup/gmail/mbsync-config. Replace LOGIN with your gmail username. This sets up a one-way channel from your gmail All Mail store to a local maildir. So the local maildir will be kept in sync with all gmail mail.

    MaildirStore local
    Path ~/Backup/gmail/maildir/
    
    IMAPStore gmail
    Host imap.gmail.com
    User LOGIN@gmail.com
    UseIMAPS yes
    CertificateFile ~/Backup/gmail/gmail.pem
    CertificateFile ~/Backup/gmail/thawte.pem
    
    Channel backup
    Master ":gmail:[Gmail]/All Mail"
    Slave :local:gmail
    Create Slave
    Expunge Slave
    Sync Pull
    
  • Save your gmail password in Keychain. The simplest way to do this is to login to your gmail account and have Safari remember the password.

  • Give security permission to access this password. Run this command (replace LOGIN with your gmail username) and when prompted click Always Allow to allow security access. security find-internet-password -g -aLOGIN-s www.google.com

  • Save this script as ~/Backup/gmail/backup-gmail and make it executable chmod +x backup-gmail. Replace LOGIN with your gmail username. This script uses security to retrieve your gmail password from Keychain - to avoid storing it in plain text in the config file. mbsync uses getpass to read the password directly from the TTY, so this won't work when you run backup-gmail directly from Terminal. It will work when run via launchd (see below).

    #!/bin/bash
    security find-internet-password -g  -a LOGIN -s www.google.com 2>&1 |\
        sed -n -e '1s/password: "\(.*\)"/\1/;1p' |\
        /opt/local/bin/mbsync --config ~/Backup/gmail/mbsync-config backup 2>&1 > ~/Backup/gmail/mbsync.log
    
  • Save this launchd plist as ~/Backup/gmail/com.rectalogic.gmail.backup.plist and then symlink it into your LaunchAgents directory ln -sf ~/Backup/gmail/com.rectalogic.gmail.backup.plist ~/Library/LaunchAgents. This places the backup script under launchd control and will run it every 24 hours. You can run it manually to test via launchctl start com.rectalogic.gmail.backup. Monitor ~/Backup/gmail/mbsync.log for errors.

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
     <key>Label</key>
      <string>com.rectalogic.gmail.backup</string>
     <key>OnDemand</key>
      <true/>
     <key>Program</key>
      <string>~/Backup/gmail/backup-gmail</string>
     <key>StartInterval</key>
      <integer>86400</integer>
    </dict>
    </plist>
    
Written on November 23, 2007