Backing up Atlassian Jira and Confluence (Cloud Editions)

This article discusses backing up Atlassian Jira and Confluence Cloud services, primarily for disaster recovery (DR) purposes. There are lots of ways to do this, but this article focusses on a simple, cheap and "no frills" solution, which is perhaps a minimum standard anyone using these products should consider.

What's the point? Do I need to? Why?

You definitely need to take disaster recovery (DR) backups of your Atlassian Jira and Confluence Cloud services. Atlassian DO NOT back up customer data - they only back up their own data. As such, if Atlassian loses the servers your data is on, then there's a good chance your data is gone too. Likewise, if your admin makes a mistake and deletes your data, then there's no other way to get it back.

Note though, that DR backups are an "all or nothing" solution. If you delete your favourite Wiki page, the DR backup can either restore the entire instance back to the way it was, or it can't do anything. You can't selectively restore pages or tickets or whatever else.

Some Cautionary Notes

• DR backups are not selective, you can't restore individual pages or tickets. You can either restore absolutely everything, or nothing at all.
• Restoring backups is not necessarily as smooth and easy as perhaps it should be, or perhaps how you expect.
• You should test your backups by performing a restore into a 'sandbox', and you should probably do this periodically to ensure your backups are still useful in the future!

Also please note that Atlassian have some very stingy resource limits on backups. During development you can expect to hit limits which prevent you doing any further testing for 72 hours or whatever. If you're planning to set something up, be ready for a fairly long job(!).

Before you start

Before you can start, you will need:

• The email address of a user that has suitable priviliges to access all the data in Confluence.
• That user needs to generate an API Token, which you will also need
• The instance name of your Confluence service

Taking Confluence Backups

You can take a Confluence backup by making a Curl call:

BKPMSG=$(curl -s -u ${EMAIL}:${API_TOKEN} -H "X-Atlassian-Token: no-check" -H "X-Requested-With: XMLHttpRequest" -H "Content-Type: application/json"  -X POST "https://${INSTANCE}/wiki/rest/obm/1.0/runbackup" -d "{\"cbAttachments\":\"true\" }" )

This call starts a background process in Atlassian, but will return some sort of output into the ${BKPMSG} variable. You can check the message looks like it was successful with:

if [ "$(echo "$BKPMSG" | grep -ic backup)" -ne 0 ]; then
  echo "BACKUP FAILED!! Message returned: $BKPMSG"
  exit 1
fi

You can then ask for progress on your backup with this:

PROGRESS_JSON=$(curl -s -u ${EMAIL}:${API_TOKEN} https://${INSTANCE}/wiki/rest/obm/1.0/getprogress.json)

You'll need to call this repeatedly (perhaps once every 30 seconds) and inspect the JSON data for the progress. During the backup, you can expect JSON a bit like this:

{"size":0,"currentStatus":"Archiving attachments.","alternativePercentage":"Estimated progress: 48%","concurrentBackupInProgress":false,"time":0,"isOutdated":false}

When the backup has completed, you can expect some JSON like this:

{"fileName":"temp/filestore/87143d25-947a-49ea-a095-12744b9f2ceb","size":3561346445,"currentStatus":"Archiving attachments.","alternativePercentage":"100%","concurrentBackupInProgress":false,"time":0,"isOutdated":false}

The main things here are no errors are reported, and a fileName has been generated. Once this has happened, you can download the file, something like this:

curl -s -L -u ${EMAIL}:${API_TOKEN} "https://${INSTANCE}/wiki/download/$FILE_NAME" -o my_new_backup.zip

Once you've downloaded the file, you can do whatever you like with it. For example, you could keep a few of them on the server you're running on, or we'd generally recommend you copy it to AWS S3 or Google bucket storage.

Taking a Jira backup

As above, you can start a backup with a Curl call:

BKPMSG=$(curl -s -u "${EMAIL}:${API_TOKEN}" -H "Accept: application/json" -H "Content-Type: application/json" --data-binary "{\"cbAttachments\":\"true\", \"exportToCloud\":\"true\"}" -X POST https://${INSTANCE}/rest/backup/1/export/runbackup )

Checking the output is similar to Confluence, but we had some other edge cases:

if [ "$(echo "$BKPMSG" | grep -ic error)" -ne 0 ]; then
  echo "BACKUP FAILED!! Message returned: $BKPMSG"
  exit 1
fi

if [ "${BKPMSG}" = "" ]; then
  echo "Response message was empty - backup progress is unknown - cannot continue"
  exit 1
fi

For Jira, we need to get the Task ID from the output in order to ask for status later:

TASK_ID=$(echo "$BKPMSG" | sed -n 's/.*"taskId"[ ]_:[ ]_"\([^"]\_\).\*/\1/p')

Now we can ask for progress of our particular task:

PROGRESS_JSON=$(curl -s -u ${EMAIL}:${API_TOKEN} -X GET https://${INSTANCE}/rest/backup/1/export/getProgress?taskId=${TASK_ID})

As with Confluence, you can expect output like this during the backup:

{"status":"InProgress","description":"Cloud Export task","message":"Compressing data & attachments","progress":50,"exportType":"CLOUD"}

...and when the backup is finished, something like this:

{"status":"Success","description":"Cloud Export task","message":"Completed export","result":"export/download/?fileId=f1b751fe-f3d7-57f1-c78b-aeb3944ea642","progress":100,"exportType":"CLOUD"}

Once the backup has completed, the result field gives us the information we need to download the backup:

curl -s -L -u ${EMAIL}:${API_TOKEN} -X GET "https://${INSTANCE}/plugins/servlet/${FILE_NAME}" -o my_jira_backup.zip

And once again, you're now free to do whatever you like with the file that was downloaded.

Putting it all together

Once you've got something working, you'll need somewhere to actually run it. Where that will be depends on your specific circumstances, but we've found that running it on a CI service is quite useful. You can schedule it to run as you like, it keeps logs of the progress its making and you can have it alert you if there's a problem. Alternatively, find somewhere else to run it, but make sure it does run, and make sure it actually gets a proper backup file each time.

As a reminder, we said above that you should also try restoring your backups to a 'sandbox' from time to time. There doesn't look to be an automated way to do this, so perhaps something to put in the diary to do periodically.

If you need help getting your cloud services under control, need your Jira/Confluence wrangling a bit or need any help with other backups or general systems maintainance, please contact us - we can help you figure out what you need and make it work for you.

Credits

We got a lot of this information from here: https://bitbucket.org/atlassianlabs/automatic-cloud-backup/src/master/ (and through quite a lot of trial and error).

Image credit: https://flic.kr/p/2ndrNd9