Skip to content

Instantly share code, notes, and snippets.

@portante

portante/DiskFile Class Usage

Last active December 16, 2015 19:50
Show Gist options
  • Save portante/5488238 to your computer and use it in GitHub Desktop.
Save portante/5488238 to your computer and use it in GitHub Desktop.
Download ZIP
OpenStack Swift DiskFile class API and usage summary comparison for DiskFile Refactoring. This is intended to be viewed "raw" so that one can see the three columns allowing for a visual comparison of the behaviors. Like methods are lined up to show as much parallelism in behavior as possible.
Raw
DiskFile Class Usage
DiskFile Class API Comparison
=============================
(as of master bfcf72a Merge "Remove keep_data_fp argument from (https://review.openstack.org/35381, Patch Set 16) (https://review.openstack.org/43973, Patch Set 1)
DiskFile constructor")
---------------------------------------------------------------+----------------------------------------------------------------+----------------------------------------------------------------
*DiskFileManager(conf)
Methods:
.pickle_async_update()
.get_diskfile()
.get_hashes()
Attributes:
.devices
.logger
.disk_chunk_size
.keep_cache_size
.bytes_per_sync
* Note that the manager is created when the controller
is created, so it is also a singleton.
DiskFile(a,c,o,keep_data_fp=) DiskFile(a,c,o) DiskFile(a,c,o)
Methods: Methods: Methods:
*.__iter__()
.close(verify_file=)
.is_deleted()
.is_expired()
.quarantine()
.get_data_file_size()
.open() .open(keep_cache_size)
.read_metadata() .read_metadata(verify_data_file_size=)
.create() .create() .create()
.write_metadata() .update_metadata()
.delete() .delete() .delete()
Attributes: Attributes: Attributes:
.quarantined_dir
.keep_cache
.metadata
*DiskFileReader() *DiskReader()
Methods: Methods:
.__iter__() .__iter__()
.get_obj_size()
.get_metadata()
.close() .close()
Attributes: Attributes:
+.was_quarantined .quarantined_dir
DiskWriter() DiskFileWriter() DiskWriter()
Methods: Methods: Methods:
.write() .write() .write()
.put() .put() .put()
* Note that the DiskFile class implements all the * Note that the DiskFileReader() object returned by the * Note that the DiskReader() object returned by the
methods necessary for a WSGI app iterator reader() method implements all the methods necessary open method implements all the methods necessary
for a WSGI app iterator for a WSGI app iterator
+ Note that if the auditor is refactored to not use the
DiskFile class, see https://review.openstack.org/44787
then we don't need the was_quarantined attribute
DiskFile Class Usage for Object Server Code Comparison
======================================================
(as of master 8a255a3 Merge "Refactor finalize_put as an (https://review.openstack.org/35381, Patch Set 11) (https://review.openstack.org/43973, Patch Set 1)
object method")
---------------------------------------------------------------+----------------------------------------------------------------+----------------------------------------------------------------
Auditor: Auditor: Auditor:
df = DiskFile(a,c,o) df = DiskFile(a,c.o) df = DiskFile(a,c,o)
df.open() with df.open(): reader = df.open()
df.get_data_file_size() metadata = odf.get_metadata() reader.get_obj_size()
reader = odf.reader()
with closing(reader):
for chunk in df: for chunk in reader: for chunk in reader:
... ... ...
df.close()
df.quarantined_dir reader.was_quarantined reader.quarantined_dir
df.close(verify_file=False) *reader.close() reader.close()
* This review highlighted that this call is missing
in Patch Set 8.
POST: POST: POST:
disk_file = DiskFile(a,c,o) disk_file = DiskFile(a,c,o) disk_file = DiskFile(a,c,o)
with disk_file.open():
disk_file.is_deleted()
disk_file.is_expired()
disk_file.get_data_file_size()
disk_file.quarantine()
disk_file.get_metadata() disk_file.read_metadata() disk_file.read_metadata()
disk_file.put_metadata() disk_file.write_metadata() disk_file.update_metadata()
PUT: PUT: PUT:
disk_file = DiskFile(a,c,o) disk_file = DiskFile(a,c,o) disk_file = DiskFile(a,c,o)
with disk_file.open():
disk_file.get_metadata() disk_file.read_metadata() disk_file.read_metadata(verify_data_file_size=False)
with disk_file.create() as writer: with df.create() as writer: with disk_file.create() as writer:
writer.write() writer.write() writer.write()
writer.disk_file.bytes_per_sync writer._disk_file.bytes_per_sync writer.bytes_per_sync
writer.put() metadata = writer.put(metadata) *writer.put(metadata)
writer.disk_file.name writer._disk_file._name writer.name
writer.disk_file.datadir writer._disk_file._datadir writer.datadir
writer.disk_file.datadir writer._disk_file._datadir writer.datadir
writer.disk_file.metadata - <write> writer._disk_file.metadata - <write> writer._metadata = metadata
writer._timestamp = nts(metadata[ts])
<ctxmgr>writer.unlinkold()
* Note that the response is constructed using the
the metadata dictionary originally passed to the
writer.put() method, which does not include any
updates made by
GET: GET: GET:
disk_file = DiskFile(a,c,o,iter_hook=sleep)) disk_file = DiskFile(a,c,o) disk_file = DiskFile(a,c,o)
disk_file.open() with disk_file.open(): reader = disk_file.open(keep_cache_size)
disk_file.is_deleted() metadata = odf.get_metadata() reader.get_metadata()
disk_file.is_expired() reader.get_obj_size()
disk_file.get_data_file_size()
disk_file.quarantine()
disk_file.get_metadata()
disk_file.close() reader.close()
disk_file.close() reader.close()
disk_file.close() reader.close()
disk_file.close() reader.close()
disk_file.keep_cache = True
<app_iter> = disk_file <app_iter> = odf.reader() <app_iter> = reader
HEAD: HEAD: HEAD:
disk_file = DiskFile(a,c,o) disk_file = DiskFile(a,c,o) DiskFile(a,c,o)
with disk_file.open():
disk_file.is_deleted() reader = disk_file.open()
disk_file.is_expired() reader.get_obj_size()
disk_file.get_data_file_size() disk_file.read_metadata() reader.get_metadata()
disk_file.quarantine()
disk_file.get_metadata()
DELETE: DELETE: DELETE:
disk_file = DiskFile(a,c,o) disk_file = DiskFile(a,c,o) DiskFile(a,c,o)
with disk_file.open(): try:
disk_file.get_metadata() disk_file.read_metadata() disk_file.read_metadata(verify_data_file_size=False)
disk_file.is_deleted() except NotFound:
disk_file.is_expired() # is-expired or is-deleted
disk_file.delete() disk_file.delete() disk_file.delete()
REPLICATE: REPLICATE: REPLICATE:
dfm = DiskFileManager()
dfm.get_hashes(device, partition, suffixes)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment