Skip to content

Instantly share code, notes, and snippets.

@benmccann
Last active December 14, 2015 17:59
Show Gist options
  • Save benmccann/5126039 to your computer and use it in GitHub Desktop.
Save benmccann/5126039 to your computer and use it in GitHub Desktop.
patch to update the wiki for OpenStack Storage support
diff --git a/Splitter.md b/Splitter.md
index 6e1ce90..d769e13 100644
--- a/Splitter.md
+++ b/Splitter.md
@@ -25,7 +25,7 @@ This can be disabled using the `--no-splitter` option with the [Generator](Gener
What does it do?
----------------
-It removes the file size constraints of remote storage locations such as Amazon S3, Rackspace Cloud Files, Ninefold and Dropbox. For example, you cannot store files larger than 5GB in size on Amazon S3. You can now configure backup with `split_into_chunks_of(5000)` to tell Backup to split the entire backup file in to chunks of 5000 megabytes (5gb) before storing it to Amazon S3. So assuming your backup is 15GB in size, it will split the backup file in 3 chunks and upload them separately.
+It removes the file size constraints of remote storage locations such as Amazon S3, OpenStack Storage, Rackspace Cloud Files, Ninefold and Dropbox. For example, you cannot store files larger than 5GB in size on Amazon S3. You can now configure backup with `split_into_chunks_of(5000)` to tell Backup to split the entire backup file in to chunks of 5000 megabytes (5gb) before storing it to Amazon S3. So assuming your backup is 15GB in size, it will split the backup file in 3 chunks and upload them separately.
### Enabling backup splitting
diff --git a/Storages.md b/Storages.md
index da13dfd..719d26a 100644
--- a/Storages.md
+++ b/Storages.md
@@ -4,6 +4,7 @@ Storages
**Currently supported storages:**
- Amazon Simple Storage Service (S3)
+- OpenStack Storage (Swift)
- Rackspace Cloud Files (Mosso)
- Ninefold Cloud Storage
- Dropbox Web Service
@@ -45,6 +46,20 @@ end
You will need an Amazon AWS (S3) account. You can get one [here](http://aws.amazon.com/s3/).
+OpenStack Storage (Swift)
+-----------------------------
+
+``` rb
+store_with OpenStack do |os|
+ os.api_key = 'my_api_key'
+ os.username = 'my_username'
+ os.container = 'my_container'
+ os.path = '/path/to/my/backups'
+ os.keep = 5
+ os.auth_url = 'https://dal05.objectstorage.softlayer.net/auth/v1.0'
+end
+```
+
Rackspace Cloud Files (Mosso)
-----------------------------
diff --git a/Syncers.md b/Syncers.md
index bd0d7e1..b1b0aa5 100644
--- a/Syncers.md
+++ b/Syncers.md
@@ -5,6 +5,7 @@ Syncers
* RSync
* Amazon S3
+* OpenStack Storage
* Rackspace Cloud Files
Storages vs Syncers
@@ -219,6 +220,55 @@ perform 50 operations at a time. Operations include:
So when Backup is busy mirroring your specified directories, it will perform 50 transfer/remove operations at the same time. It's very fast!
+OpenStack Storage
+---------------------
+
+``` rb
+sync_with Cloud::OpenStack do |os|
+ os.username = "my_username"
+ os.api_key = "my_api_key"
+ os.container = "my_container"
+ os.auth_url = "https://dal05.objectstorage.softlayer.net/auth/v1.0"
+ os.path = "/backups"
+ os.mirror = true
+ os.concurrency_type = :threads
+ os.concurrency_level = 50
+
+ os.directories do |directory|
+ directory.add "/path/to/directory/to/sync"
+ directory.add "/path/to/other/directory/to/sync"
+ end
+end
+```
+
+**Additional Notes**
+
+**NOTE** Unlike RSync, which has the ability to transfer only parts of individual files, OpenStack Storage does not support this
+and will have to transfer a whole file in order to update it. However, OpenStack Storage does provide "checksums". This allows Backup to check
+if a file you synced up to OpenStack Storage has or has not been updated locally since then. In the case that it has not changed, Backup will
+not re-upload it (and thus, save you bandwidth/system resources).
+
+When enabling mirroring (`cf.mirror = true`) Backup will keep an exact one-on-one mirror from your filesystem on OpenStack Storage. This means
+that if you add a file to one of the mirrored directories on your filesystem, it will sync it to OpenStack Storage. If you remove a file from
+one of these directories, it will also remove it from OpenStack Storage.
+
+The `directory.add` method allows you to add any directories you want to sync from your filesystem to your OpenStack Storage account.
+
+You can use the `concurrency_type` and `concurrency_level` to increase concurrency on file transfers. The recommended `concurrency_type` is `:threads`.
+Threads do not require additional memory to spin up, and for operations such as file transfers, they are absolutely excellent in parallel performance, so it is highly encouraged to
+use them. Processes on the other hand are discouraged, they consume a lot of ram and are likely to cause problems if your machine doesn't have a lot of memory
+and you spin up a bunch of processes to handle it. You may want to change the `concurrency_level` (higher or lower, feel free to experiment),
+but we recommend using `:threads` and not `:processes` when doing so.
+
+For each `concurrency_level` you can handle one additional file concurrently. This means that if you set `concurrency_level = 50`, it means Backup will
+perform 50 operations at a time. Operations include:
+
+- Transfering a single file
+- Removing a single file
+
+So when Backup is busy mirroring your specified directories, it will perform 50 transfer/remove operations at the same time. It's very fast!
+
+
Rackspace Cloud Files
---------------------
@@ -248,33 +298,10 @@ end
- https://auth.api.rackspacecloud.com (US - Default)
- https://lon.auth.api.rackspacecloud.com (UK)
-**NOTE** Unlike RSync, which has the ability to transfer only parts of individual files, Rackspace Cloud Files does not support this
-and will have to transfer a whole file in order to update it. However, Rackspace Cloud Files does provide "checksums". This allows Backup to check
-if a file you synced up to Rackspace Cloud Files has or has not been updated locally since then. In the case that it has not changed, Backup will
-not re-upload it (and thus, save you bandwidth/system resources).
+**NOTE** All the notes from OpenStack Storage apply to RackSpace Cloud Files since RackSpace is an OpenStack provider.
Set `servicenet = true` if Backup runs on a Rackspace server. It will avoid transfer charges and it's more performant.
-When enabling mirroring (`cf.mirror = true`) Backup will keep an exact one-on-one mirror from your filesystem on Rackspace Cloud Files. This means
-that if you add a file to one of the mirrored directories on your filesystem, it will sync it to Rackspace Cloud Files. If you remove a file from
-one of these directories, it will also remove it from Rackspace Cloud Files.
-
-The `directory.add` method allows you to add any directories you want to sync from your filesystem to your Rackspace Cloud Files account.
-
-You can use the `concurrency_type` and `concurrency_level` to increase concurrency on file transfers. The recommended `concurrency_type` is `:threads`.
-Threads do not require additional memory to spin up, and for operations such as file transfers, they are absolutely excellent in parallel performance, so it is highly encouraged to
-use them. Processes on the other hand are discouraged, they consume a lot of ram and are likely to cause problems if your machine doesn't have a lot of memory
-and you spin up a bunch of processes to handle it. You may want to change the `concurrency_level` (higher or lower, feel free to experiment),
-but we recommend using `:threads` and not `:processes` when doing so.
-
-For each `concurrency_level` you can handle one additional file concurrently. This means that if you set `concurrency_level = 50`, it means Backup will
-perform 50 operations at a time. Operations include:
-
-- Transfering a single file
-- Removing a single file
-
-So when Backup is busy mirroring your specified directories, it will perform 50 transfer/remove operations at the same time. It's very fast!
-
Default Configuration for the RSync Syncer
==========================================
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment