openstack swift notes.

swift.md

Requirements

• Ubuntu 12.04
• OpenStack havana release
• python-swiftclient Swifit CLI
• cURL
• Swift tools such as Swift-Recon, Swift-Informant, and Swift-Dispersion
• A StatsD server. https://github.com/etsy/statsd

CAP theorem

• Consistency
• Availability
• Partial Tolerance

Cloud Gateway

• provide file or block interfaces to an anpplication (for example, CIFS, NFS, iSCSI, or FTP/SFTP) and perform conversion to the cloud.

OpenStack Swift

Openstack swift is the magic that converts a set of unconnected commodity servers into a scalable, durable, easy-to-manage storage system.

• Open source
• Open Standard
• Account/container/object structure
• Global cluster capability
• Partial Object retrieval
• Middleware architecture
• Large object support
• Additional functionality
  • object versioning
  • expiring objects
  • rate limiting
  • temporary URL support
  • CNAME lookup
  • domain remap
  • static web mode

OpenStack Swift Architecture.

logical organization of objects.

• A Tenant (Account) is assigned an account. The account holds containers. Each container holds objects (files).
• A tenant can create additional user.

Key architectural principles

• Masterless
• Loosely coupled
• Load spreading
• Self-healing
• Data organizing a number of object storage systems simply return a hash key for a submitted object and provide a completely flat namespace.
• Available and eventually consistent.

Physical data organization

• Region (geographically seperated)
• Zone (different availability: different physical building, power sources, or network connection)
• storage server
• Disk

swift will store a number of replicas(default = 3) of an object onto different disks.

• As-unique-as-possible algorithm. (as far away as possible in terms of region/zone/server/disk)
• ring builder and partition.
• account and container information is kept in SQLite.

Data path software servers

• Proxy server: accept HTTP request from a user.
• Account server: track names of containers in a particular account.
• Container server: deal with object names in a particular container.
• Object Server: store objects.

CRUD

basic operations:

• create All copies of the object are written to the local region.(Write affinity). The object is then asynchronously moved to other regions. A dedicated replication network may be used for this operation.
• read The read request is issued to object servers in the same region as the proxy server. (Read affinity)
• update
• delete Treated like an update but instead of a new version. (Tombstone)

##Post-processing software component

• Replication Replication uses rsync.
• Updaters object updater and container updater.
• Auditors self healing.
• Other processes.
  • Acount reaper
  • Object expirer
  • Drive audit
  • container to container synchronization.

Middleware modules

• Auth
  • Auth token
  • container level ACL. (Access control list)
• Logging
• Health check Monitor if proxy server alive.
• Domain remap remap account/container name to host domain name.
• CNAME lookup create domain name and map to account/container
• Rate limiting limit rate of request.
• container and account quotas
• Bulk delete
• Bulk archive auto-extraction
• TempURL Access not authenticated but expires after a certain duration of time.
• Swift origin server
• Static web Web presentation of object structure.
• Form post Upload object in web form.
• Recon Monitoring

Additional feature.

• Large object support. virtually unlimited size objects. segmentation.
• Metadata (object/container/account attributes)
• Multirange support. partial read.
• CORS cross domain access.
• Server-side copies copy an object on server.
• Cluster health tool swift-dispersion-report

Installing OpenStack Swift

advanced users may want to use utilities such puppet or Chef to simplify the process.