Skip to content

Instantly share code, notes, and snippets.

@NikosAlexandris

NikosAlexandris/store_attributes

Created November 11, 2023 15:59
Show Gist options
  • Save NikosAlexandris/7272d8e4ac3ad3b51d9ad5459cfbac2f to your computer and use it in GitHub Desktop.
Save NikosAlexandris/7272d8e4ac3ad3b51d9ad5459cfbac2f to your computer and use it in GitHub Desktop.
Download ZIP
Store attributes before and after functions run in SingleHdf5ToZarr.translate()
Raw
store_attributes
- Attribute name _nc3_strict with value 1
- Attribute name _NCProperties with value b'version=2,netcdf=4.7.3,hdf5=1.10.6,'
- Attribute name Conventions with value b'CF-1.7,ACDD-1.3'
+ Attribute value = 'CF-1.7,ACDD-1.3' will be fed to n = 'Conventions' in 'zobj.attrs[Conventions]'
- Attribute name id with value b'DOI:10.5676/EUM_SAF_CM/SARAH/V003'
+ Attribute value = 'DOI:10.5676/EUM_SAF_CM/SARAH/V003' will be fed to n = 'id' in 'zobj.attrs'
- Attribute name product_version with value b'3.0'
+ Attribute value = '3.0' will be fed to n = 'product_version' in 'zobj.attrs'
- Attribute name institution with value b'EUMETSAT/CMSAF'
+ Attribute value = 'EUMETSAT/CMSAF' will be fed to n = 'institution' in 'zobj.attrs'
- Attribute name creator_name with value b'DE/DWD'
+ Attribute value = 'DE/DWD' will be fed to n = 'creator_name' in 'zobj.attrs'
- Attribute name creator_email with value b'[email protected]'
+ Attribute value = '[email protected]' will be fed to n = 'creator_email' in 'zobj.attrs'
- Attribute name creator_url with value b'http://www.cmsaf.eu/'
+ Attribute value = 'http://www.cmsaf.eu/' will be fed to n = 'creator_url' in 'zobj.attrs'
- Attribute name creator_type with value b'institution'
+ Attribute value = 'institution' will be fed to n = 'creator_type' in 'zobj.attrs'
- Attribute name publisher_name with value b'EUMETSAT/CMSAF'
+ Attribute value = 'EUMETSAT/CMSAF' will be fed to n = 'publisher_name' in 'zobj.attrs'
- Attribute name publisher_email with value b'[email protected]'
+ Attribute value = '[email protected]' will be fed to n = 'publisher_email' in 'zobj.attrs'
- Attribute name publisher_url with value b'http://www.cmsaf.eu/'
+ Attribute value = 'http://www.cmsaf.eu/' will be fed to n = 'publisher_url' in 'zobj.attrs'
- Attribute name publisher_type with value b'institution'
+ Attribute value = 'institution' will be fed to n = 'publisher_type' in 'zobj.attrs'
- Attribute name references with value b'https://doi.org/10.5676/EUM_SAF_CM/SARAH/V003'
+ Attribute value = 'https://doi.org/10.5676/EUM_SAF_CM/SARAH/V003' will be fed to n = 'references' in 'zobj.attrs'
- Attribute name keywords_vocabulary with value b'GCMD Science Keywords, Version 8.6'
+ Attribute value = 'GCMD Science Keywords, Version 8.6' will be fed to n = 'keywords_vocabulary' in 'zobj.attrs'
- Attribute name keywords with value b'EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > INCOMING SOLAR RADIATION'
+ Attribute value = 'EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > INCOMING SOLAR RADIATION' will be fed to n = 'keywords' in 'zobj.attrs'
- Attribute name project with value b'Satellite Application Facility on Climate Monitoring (CM SAF)'
+ Attribute value = 'Satellite Application Facility on Climate Monitoring (CM SAF)' will be fed to n = 'project' in 'zobj.attrs'
- Attribute name standard_name_vocabulary with value b'Standard Name Table (v51, 16 May 2018)'
+ Attribute value = 'Standard Name Table (v51, 16 May 2018)' will be fed to n = 'standard_name_vocabulary' in 'zobj.attrs'
- Attribute name date_created with value b'2021-09-29T06:30:32Z'
+ Attribute value = '2021-09-29T06:30:32Z' will be fed to n = 'date_created' in 'zobj.attrs'
- Attribute name geospatial_lat_units with value b'degrees_north'
+ Attribute value = 'degrees_north' will be fed to n = 'geospatial_lat_units' in 'zobj.attrs'
- Attribute name geospatial_lat_min with value [-65.]
+ Attribute value = '-65.0' will be fed to n = 'geospatial_lat_min' in 'zobj.attrs'
- Attribute name geospatial_lat_max with value [65.]
+ Attribute value = '65.0' will be fed to n = 'geospatial_lat_max' in 'zobj.attrs'
- Attribute name geospatial_lon_units with value b'degrees_east'
+ Attribute value = 'degrees_east' will be fed to n = 'geospatial_lon_units' in 'zobj.attrs'
- Attribute name geospatial_lon_min with value [-65.]
+ Attribute value = '-65.0' will be fed to n = 'geospatial_lon_min' in 'zobj.attrs'
- Attribute name geospatial_lon_max with value [65.]
+ Attribute value = '65.0' will be fed to n = 'geospatial_lon_max' in 'zobj.attrs'
- Attribute name geospatial_lat_resolution with value b'0.05 degree'
+ Attribute value = '0.05 degree' will be fed to n = 'geospatial_lat_resolution' in 'zobj.attrs'
- Attribute name geospatial_lon_resolution with value b'0.05 degree'
+ Attribute value = '0.05 degree' will be fed to n = 'geospatial_lon_resolution' in 'zobj.attrs'
- Attribute name time_coverage_duration with value b'P1D'
+ Attribute value = 'P1D' will be fed to n = 'time_coverage_duration' in 'zobj.attrs'
- Attribute name time_coverage_resolution with value b'PT30M'
+ Attribute value = 'PT30M' will be fed to n = 'time_coverage_resolution' in 'zobj.attrs'
- Attribute name time_coverage_start with value b'2020-01-02T00:00:00'
+ Attribute value = '2020-01-02T00:00:00' will be fed to n = 'time_coverage_start' in 'zobj.attrs'
- Attribute name time_coverage_end with value b'2020-01-03T00:00:00'
+ Attribute value = '2020-01-03T00:00:00' will be fed to n = 'time_coverage_end' in 'zobj.attrs'
- Attribute name platform_vocabulary with value b'GCMD Platforms, Version 8.6'
+ Attribute value = 'GCMD Platforms, Version 8.6' will be fed to n = 'platform_vocabulary' in 'zobj.attrs'
- Attribute name platform with value b'Earth Observation Satellites > METEOSAT'
+ Attribute value = 'Earth Observation Satellites > METEOSAT' will be fed to n = 'platform' in 'zobj.attrs'
- Attribute name instrument_vocabulary with value b'GCMD Instruments, Version 8.6'
+ Attribute value = 'GCMD Instruments, Version 8.6' will be fed to n = 'instrument_vocabulary' in 'zobj.attrs'
- Attribute name instrument with value b'SEVIRI > Spinning Enhanced Visible and Infrared Imager'
+ Attribute value = 'SEVIRI > Spinning Enhanced Visible and Infrared Imager' will be fed to n = 'instrument' in 'zobj.attrs'
- Attribute name variable_id with value b'SIS'
+ Attribute value = 'SIS' will be fed to n = 'variable_id' in 'zobj.attrs'
- Attribute name license with value b'The CM SAF data are owned by EUMETSAT and are available to all users free of charge and with no conditions to use. If you wish to use these products,
EUMETSAT\'s copyright credit must be shown by displaying the words "Copyright (c) (2022) EUMETSAT" under/in each of these SAF Products used in a project or shown in a publication or
website.\n\nPlease follow the citation guidelines given at https://doi.org/10.5676/EUM_SAF_CM/SARAH/V003 and also register as a user at http://cm-saf.eumetsat.int/ to receive latest
information on CM SAF services and to get access to the CM SAF User Help Desk.'
+ Attribute value = 'The CM SAF data are owned by EUMETSAT and are available to all users free of charge and with no conditions to use. If you wish to use these products, EUMETSAT's
copyright credit must be shown by displaying the words "Copyright (c) (2022) EUMETSAT" under/in each of these SAF Products used in a project or shown in a publication or website.
Please follow the citation guidelines given at https://doi.org/10.5676/EUM_SAF_CM/SARAH/V003 and also register as a user at http://cm-saf.eumetsat.int/ to receive latest information on CM SAF
services and to get access to the CM SAF User Help Desk.' will be fed to n = 'license' in 'zobj.attrs'
- Attribute name title with value b'CM SAF Surface Solar Radiation Climate Data Record using Heliosat, Edition 3 (SARAH-3)'
+ Attribute value = 'CM SAF Surface Solar Radiation Climate Data Record using Heliosat, Edition 3 (SARAH-3)' will be fed to n = 'title' in 'zobj.attrs'
- Attribute name summary with value b'This file contains data from the CM SAF Surface Solar Radiation Climate Data Record using Heliosat, Edition 3'
+ Attribute value = 'This file contains data from the CM SAF Surface Solar Radiation Climate Data Record using Heliosat, Edition 3' will be fed to n = 'summary' in 'zobj.attrs'
Detailed store contents:
zmetadata: {'.zgroup': {'zarr_format': 2}, '.zattrs': {'Conventions': 'CF-1.7,ACDD-1.3', 'summary': 'This file contains data from the CM SAF Surface Solar Radiation Climate Data Record using
Heliosat, Edition 3'}}
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "SIS": shape (48, 2600, 2600), type "<i2">,
'units': 'W m-2',
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
> za is before _transfer_attrs :
<zarr.attrs.Attributes object at 0x7f3fd6681b50>
- Attribute name _FillValue with value [-999]
- Attribute name missing_value with value [-999]
+ Attribute value = '-999' will be fed to n = 'missing_value' in 'zobj.attrs'
- Attribute name standard_name with value b'surface_downwelling_shortwave_flux_in_air'
+ Attribute value = 'surface_downwelling_shortwave_flux_in_air' will be fed to n = 'standard_name' in 'zobj.attrs'
- Attribute name long_name with value b'Surface Downwelling Shortwave Radiation'
+ Attribute value = 'Surface Downwelling Shortwave Radiation' will be fed to n = 'long_name' in 'zobj.attrs'
- Attribute name units with value b'W m-2'
+ Attribute value = 'W m-2' will be fed to n = 'units' in 'zobj.attrs'
- Attribute name cell_methods with value b'time: point'
+ Attribute value = 'time: point' will be fed to n = 'cell_methods' in 'zobj.attrs'
- Attribute name DIMENSION_LIST with value [array([<HDF5 object reference>], dtype=object)
array([<HDF5 object reference>], dtype=object)
array([<HDF5 object reference>], dtype=object)]
>> All attributes in za after _transfer_attrs:
<zarr.attrs.Attributes object at 0x7f3fd6681b50>
>> 'units' attribute not found in za.attrs
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "bnds": shape (2,), type ">f4">,
'units': None,
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
For h5obj = <HDF5 dataset "bnds": shape (2,), type ">f4"> I will do `fill = None`
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "lat": shape (2600,), type "<f4">,
'units': 'degrees_north',
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
For h5obj = <HDF5 dataset "lat": shape (2600,), type "<f4"> I will do `fill = None`
> za is before _transfer_attrs :
<zarr.attrs.Attributes object at 0x7f3fd643c750>
- Attribute name CLASS with value b'DIMENSION_SCALE'
- Attribute name NAME with value b'lat'
- Attribute name _Netcdf4Dimid with value 3
- Attribute name standard_name with value b'latitude'
+ Attribute value = 'latitude' will be fed to n = 'standard_name' in 'zobj.attrs'
- Attribute name long_name with value b'latitude'
+ Attribute value = 'latitude' will be fed to n = 'long_name' in 'zobj.attrs'
- Attribute name units with value b'degrees_north'
+ Attribute value = 'degrees_north' will be fed to n = 'units' in 'zobj.attrs'
- Attribute name axis with value b'Y'
+ Attribute value = 'Y' will be fed to n = 'axis' in 'zobj.attrs'
- Attribute name bounds with value b'lat_bnds'
+ Attribute value = 'lat_bnds' will be fed to n = 'bounds' in 'zobj.attrs'
- Attribute name REFERENCE_LIST with value [(<HDF5 object reference>, 0) (<HDF5 object reference>, 1)]
>> All attributes in za after _transfer_attrs:
<zarr.attrs.Attributes object at 0x7f3fd643c750>
>> 'units' attribute not found in za.attrs
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "lat_bnds": shape (2600, 2), type "<f4">,
'units': None,
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
> za is before _transfer_attrs :
<zarr.attrs.Attributes object at 0x7f3fd641cc10>
- Attribute name DIMENSION_LIST with value [array([<HDF5 object reference>], dtype=object)
array([<HDF5 object reference>], dtype=object)]
>> All attributes in za after _transfer_attrs:
<zarr.attrs.Attributes object at 0x7f3fd641cc10>
>> 'units' attribute not found in za.attrs
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "lon": shape (2600,), type "<f4">,
'units': 'degrees_east',
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
For h5obj = <HDF5 dataset "lon": shape (2600,), type "<f4"> I will do `fill = None`
> za is before _transfer_attrs :
<zarr.attrs.Attributes object at 0x7f3fd64141d0>
- Attribute name CLASS with value b'DIMENSION_SCALE'
- Attribute name NAME with value b'lon'
- Attribute name _Netcdf4Dimid with value 1
- Attribute name standard_name with value b'longitude'
+ Attribute value = 'longitude' will be fed to n = 'standard_name' in 'zobj.attrs'
- Attribute name long_name with value b'longitude'
+ Attribute value = 'longitude' will be fed to n = 'long_name' in 'zobj.attrs'
- Attribute name units with value b'degrees_east'
+ Attribute value = 'degrees_east' will be fed to n = 'units' in 'zobj.attrs'
- Attribute name axis with value b'X'
+ Attribute value = 'X' will be fed to n = 'axis' in 'zobj.attrs'
- Attribute name bounds with value b'lon_bnds'
+ Attribute value = 'lon_bnds' will be fed to n = 'bounds' in 'zobj.attrs'
- Attribute name REFERENCE_LIST with value [(<HDF5 object reference>, 0) (<HDF5 object reference>, 2)]
>> All attributes in za after _transfer_attrs:
<zarr.attrs.Attributes object at 0x7f3fd64141d0>
>> 'units' attribute not found in za.attrs
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "lon_bnds": shape (2600, 2), type "<f4">,
'units': None,
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
> za is before _transfer_attrs :
<zarr.attrs.Attributes object at 0x7f3fd6408c10>
- Attribute name DIMENSION_LIST with value [array([<HDF5 object reference>], dtype=object)
array([<HDF5 object reference>], dtype=object)]
>> All attributes in za after _transfer_attrs:
<zarr.attrs.Attributes object at 0x7f3fd6408c10>
>> 'units' attribute not found in za.attrs
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "record_status": shape (48,), type "|i1">,
'units': None,
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
> za is before _transfer_attrs :
<zarr.attrs.Attributes object at 0x7f3fd6402bd0>
- Attribute name comment with value b'Overall status of each record (timestamp) in this file. If a record is flagged as not ok, it is recommended not to use it.'
+ Attribute value = 'Overall status of each record (timestamp) in this file. If a record is flagged as not ok, it is recommended not to use it.' will be fed to n = 'comment' in 'zobj.attrs'
- Attribute name flag_meanings with value b'ok void bad_quality'
+ Attribute value = 'ok void bad_quality' will be fed to n = 'flag_meanings' in 'zobj.attrs'
- Attribute name long_name with value b'Record Status'
+ Attribute value = 'Record Status' will be fed to n = 'long_name' in 'zobj.attrs'
- Attribute name flag_values with value [0 1 2]
+ Attribute value = '[0, 1, 2]' will be fed to n = 'flag_values' in 'zobj.attrs'
- Attribute name DIMENSION_LIST with value [array([<HDF5 object reference>], dtype=object)]
>> All attributes in za after _transfer_attrs:
<zarr.attrs.Attributes object at 0x7f3fd6402bd0>
>> 'units' attribute not found in za.attrs
/software/kerchunk/.kerchunk_virtual_environment/lib/python3.11/site-packages/kerchunk/hdf.py:671 _is_netcdf_datetime
locals(): {
'dataset': <HDF5 dataset "time": shape (48,), type "<f8">,
'units': 'hours since 1983-1-1 00:00:00',
'debug': <devtools.debug.Debug object at 0x7f3fd6862510>,
} (dict) len=3
For h5obj = <HDF5 dataset "time": shape (48,), type "<f8"> I will do `fill = None`
> za is before _transfer_attrs :
<zarr.attrs.Attributes object at 0x7f3fd63ec3d0>
- Attribute name CLASS with value b'DIMENSION_SCALE'
- Attribute name NAME with value b'time'
- Attribute name _Netcdf4Dimid with value 0
- Attribute name axis with value b'T'
+ Attribute value = 'T' will be fed to n = 'axis' in 'zobj.attrs'
- Attribute name calendar with value b'proleptic_gregorian'
+ Attribute value = 'proleptic_gregorian' will be fed to n = 'calendar' in 'zobj.attrs'
- Attribute name long_name with value b'time'
+ Attribute value = 'time' will be fed to n = 'long_name' in 'zobj.attrs'
- Attribute name standard_name with value b'time'
+ Attribute value = 'time' will be fed to n = 'standard_name' in 'zobj.attrs'
- Attribute name units with value b'hours since 1983-1-1 00:00:00'
+ Attribute value = 'hours since 1983-1-1 00:00:00' will be fed to n = 'units' in 'zobj.attrs'
- Attribute name REFERENCE_LIST with value [(<HDF5 object reference>, 0) (<HDF5 object reference>, 0)]
>> All attributes in za after _transfer_attrs:
<zarr.attrs.Attributes object at 0x7f3fd63ec3d0>
>> units in za is after _transfer_attrs : hours since 1983-1-1 00:00:00
Detailed store contents after visititems :
zmetadata: {'.zgroup': {'zarr_format': 2}, '.zattrs': {'Conventions': 'CF-1.7,ACDD-1.3', 'summary': 'This file contains data from the CM SAF Surface Solar Radiation Climate Data Record using
Heliosat, Edition 3'}, 'SIS/.zarray': {'chunks': [1, 1, 2600], 'compressor': None, 'dtype': '<i2', 'fill_value': -999, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [48,
2600, 2600], 'zarr_format': 2}, 'SIS/.zattrs': {'_ARRAY_DIMENSIONS': ['time', 'lat', 'lon'], 'missing_value': -999}, 'lat/.zarray': {'chunks': [2600], 'compressor': None, 'dtype': '<f4',
'fill_value': None, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [2600], 'zarr_format': 2}, 'lat/.zattrs': {'_ARRAY_DIMENSIONS': ['lat'], 'standard_name': 'latitude'},
'lat_bnds/.zarray': {'chunks': [2600, 2], 'compressor': None, 'dtype': '<f4', 'fill_value': 9.969209968386869e+36, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [2600, 2],
'zarr_format': 2}, 'lat_bnds/.zattrs': {'_ARRAY_DIMENSIONS': ['lat', 'bnds']}, 'lon/.zarray': {'chunks': [2600], 'compressor': None, 'dtype': '<f4', 'fill_value': None, 'filters': [{'id':
'zlib', 'level': 4}], 'order': 'C', 'shape': [2600], 'zarr_format': 2}, 'lon/.zattrs': {'_ARRAY_DIMENSIONS': ['lon'], 'standard_name': 'longitude'}, 'lon_bnds/.zarray': {'chunks': [2600, 2],
'compressor': None, 'dtype': '<f4', 'fill_value': 9.969209968386869e+36, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [2600, 2], 'zarr_format': 2}, 'lon_bnds/.zattrs':
{'_ARRAY_DIMENSIONS': ['lon', 'bnds']}, 'record_status/.zarray': {'chunks': [48], 'compressor': None, 'dtype': '|i1', 'fill_value': -127, 'filters': [{'id': 'zlib', 'level': 4}], 'order':
'C', 'shape': [48], 'zarr_format': 2}, 'record_status/.zattrs': {'_ARRAY_DIMENSIONS': ['time'], 'comment': 'Overall status of each record (timestamp) in this file. If a record is flagged as
not ok, it is recommended not to use it.'}, 'time/.zarray': {'chunks': [512], 'compressor': None, 'dtype': '<f8', 'fill_value': None, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C',
'shape': [48], 'zarr_format': 2}, 'time/.zattrs': {'_ARRAY_DIMENSIONS': ['time'], 'axis': 'T'}}
Detailed LazyReferenceMapper store contents:
zmetadata: {'.zgroup': {'zarr_format': 2}, '.zattrs': {'Conventions': 'CF-1.7,ACDD-1.3', 'summary': 'This file contains data from the CM SAF Surface Solar Radiation Climate Data Record using
Heliosat, Edition 3'}, 'SIS/.zarray': {'chunks': [1, 1, 2600], 'compressor': None, 'dtype': '<i2', 'fill_value': -999, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [48,
2600, 2600], 'zarr_format': 2}, 'SIS/.zattrs': {'_ARRAY_DIMENSIONS': ['time', 'lat', 'lon'], 'missing_value': -999}, 'lat/.zarray': {'chunks': [2600], 'compressor': None, 'dtype': '<f4',
'fill_value': None, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [2600], 'zarr_format': 2}, 'lat/.zattrs': {'_ARRAY_DIMENSIONS': ['lat'], 'standard_name': 'latitude'},
'lat_bnds/.zarray': {'chunks': [2600, 2], 'compressor': None, 'dtype': '<f4', 'fill_value': 9.969209968386869e+36, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [2600, 2],
'zarr_format': 2}, 'lat_bnds/.zattrs': {'_ARRAY_DIMENSIONS': ['lat', 'bnds']}, 'lon/.zarray': {'chunks': [2600], 'compressor': None, 'dtype': '<f4', 'fill_value': None, 'filters': [{'id':
'zlib', 'level': 4}], 'order': 'C', 'shape': [2600], 'zarr_format': 2}, 'lon/.zattrs': {'_ARRAY_DIMENSIONS': ['lon'], 'standard_name': 'longitude'}, 'lon_bnds/.zarray': {'chunks': [2600, 2],
'compressor': None, 'dtype': '<f4', 'fill_value': 9.969209968386869e+36, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C', 'shape': [2600, 2], 'zarr_format': 2}, 'lon_bnds/.zattrs':
{'_ARRAY_DIMENSIONS': ['lon', 'bnds']}, 'record_status/.zarray': {'chunks': [48], 'compressor': None, 'dtype': '|i1', 'fill_value': -127, 'filters': [{'id': 'zlib', 'level': 4}], 'order':
'C', 'shape': [48], 'zarr_format': 2}, 'record_status/.zattrs': {'_ARRAY_DIMENSIONS': ['time'], 'comment': 'Overall status of each record (timestamp) in this file. If a record is flagged as
not ok, it is recommended not to use it.'}, 'time/.zarray': {'chunks': [512], 'compressor': None, 'dtype': '<f8', 'fill_value': None, 'filters': [{'id': 'zlib', 'level': 4}], 'order': 'C',
'shape': [48], 'zarr_format': 2}, 'time/.zattrs': {'_ARRAY_DIMENSIONS': ['time'], 'axis': 'T'}}
@NikosAlexandris
Copy link
Author

New debugging effort :

> Create Zarr Array
   Equivalent to the HDF5 dataset : /time
   Metadata or top level Key : 'time/.zarray' + Value : 'b'{\n    "chunks": [\n        512\n    ],\n    "compressor": null,\n    "dtype": "<f8",\n    "fill_value": null,\n    "filters": [\n        {\n
"id": "zlib",\n            "level": 4\n        }\n    ],\n    "order": "C",\n    "shape": [\n        48\n    ],\n    "zarr_format": 2\n}''
   .Zmetadata for 'time' before : Not found
   .Zmetadata for 'time' after : Not found
 + Created Zarr Array : <zarr.core.Array '/time' (48,) float64>

   Initial attributes in <zarr.core.Array '/time' (48,) float64> are empty!

> Transferring attributes
   from HDF5 object : '<HDF5 dataset "time": shape (48,), type "<f8">'
   to Zarr object : '<zarr.core.Array '/time' (48,) float64>'
   ----------------------------------------------------------
   Processing attribute CLASS with value b'DIMENSION_SCALE' :
   Skipping hidden attribute CLASS
   Processing attribute NAME with value b'time' :
   Skipping hidden attribute NAME
   Processing attribute _Netcdf4Dimid with value 0 :
   Skipping hidden attribute _Netcdf4Dimid
   Processing attribute axis with value b'T' :
   Setting Zarr object attribute axis to T
   Metadata or top level Key : 'time/.zattrs' + Value : 'b'{\n    "axis": "T"\n}''
   .Zmetadata for 'time' before : Not found
   .Zmetadata for 'time' after : {'axis': 'T'}
   Updated Attributes after _transfer_attrs()
   attributes : <zarr.attrs.Attributes object at 0x7f2eaf506190>
     axis: T
   Processing attribute calendar with value b'proleptic_gregorian' :
   Setting Zarr object attribute calendar to proleptic_gregorian
   Metadata or top level Key : 'time/.zattrs' + Value : 'b'{\n    "axis": "T",\n    "calendar": "proleptic_gregorian"\n}''
   .Zmetadata for 'time' before : {'axis': 'T'}
   .Zmetadata for 'time' after : {'axis': 'T', 'calendar': 'proleptic_gregorian'}
   Updated Attributes after _transfer_attrs()
   attributes : <zarr.attrs.Attributes object at 0x7f2eaf506190>
     axis: T
     calendar: proleptic_gregorian
   Processing attribute long_name with value b'time' :
   Setting Zarr object attribute long_name to time
   Metadata or top level Key : 'time/.zattrs' + Value : 'b'{\n    "axis": "T",\n    "calendar": "proleptic_gregorian",\n    "long_name": "time"\n}''
   .Zmetadata for 'time' before : {'axis': 'T', 'calendar': 'proleptic_gregorian'}
   .Zmetadata for 'time' after : {'axis': 'T', 'calendar': 'proleptic_gregorian', 'long_name': 'time'}
   Updated Attributes after _transfer_attrs()
   attributes : <zarr.attrs.Attributes object at 0x7f2eaf506190>
     axis: T
     calendar: proleptic_gregorian
     long_name: time
   Processing attribute standard_name with value b'time' :
   Setting Zarr object attribute standard_name to time
   Metadata or top level Key : 'time/.zattrs' + Value : 'b'{\n    "axis": "T",\n    "calendar": "proleptic_gregorian",\n    "long_name": "time",\n    "standard_name": "time"\n}''
   .Zmetadata for 'time' before : {'axis': 'T', 'calendar': 'proleptic_gregorian', 'long_name': 'time'}
   .Zmetadata for 'time' after : {'axis': 'T', 'calendar': 'proleptic_gregorian', 'long_name': 'time', 'standard_name': 'time'}
   Updated Attributes after _transfer_attrs()
   attributes : <zarr.attrs.Attributes object at 0x7f2eaf506190>
     axis: T
     calendar: proleptic_gregorian
     long_name: time
     standard_name: time
   Processing attribute units with value b'hours since 1983-1-1 00:00:00' :
   Setting Zarr object attribute units to hours since 1983-1-1 00:00:00
   Metadata or top level Key : 'time/.zattrs' + Value : 'b'{\n    "axis": "T",\n    "calendar": "proleptic_gregorian",\n    "long_name": "time",\n    "standard_name": "time",\n    "units": "hours since 1983-1-1
00:00:00"\n}''
   .Zmetadata for 'time' before : {'axis': 'T', 'calendar': 'proleptic_gregorian', 'long_name': 'time', 'standard_name': 'time'}
   .Zmetadata for 'time' after : {'axis': 'T', 'calendar': 'proleptic_gregorian', 'long_name': 'time', 'standard_name': 'time', 'units': 'hours since 1983-1-1 00:00:00'}
   Updated Attributes after _transfer_attrs()
   attributes : <zarr.attrs.Attributes object at 0x7f2eaf506190>
     axis: T
     calendar: proleptic_gregorian
     long_name: time
     standard_name: time
     units: hours since 1983-1-1 00:00:00
   Processing attribute REFERENCE_LIST with value [(<HDF5 object reference>, 0) (<HDF5 object reference>, 0)] :
   Skipping hidden attribute REFERENCE_LIST


   Source Attributes
     CLASS: b'DIMENSION_SCALE'
     NAME: b'time'
     _Netcdf4Dimid: 0
     axis: b'T'
     calendar: b'proleptic_gregorian'
     long_name: b'time'
     standard_name: b'time'
     units: b'hours since 1983-1-1 00:00:00'
     REFERENCE_LIST: [(<HDF5 object reference>, 0) (<HDF5 object reference>, 0)]

   Updated Attributes after _transfer_attrs()
   attributes : <zarr.attrs.Attributes object at 0x7f2eaf506190>
     axis: T
     calendar: proleptic_gregorian
     long_name: time
     standard_name: time
     units: hours since 1983-1-1 00:00:00
> Get Array Dimensions
   (adims) : ['time']
   from : <HDF5 dataset "time": shape (48,), type "<f8">

> Add Array Dimensions
   Add _ARRAY_DIMENSIONS to '<zarr.core.Array '/time' (48,) float64>'
   with existing attributes : <zarr.attrs.Attributes object at 0x7f2eaf506190>

   Metadata or top level Key : 'time/.zattrs' + Value : 'b'{\n    "_ARRAY_DIMENSIONS": [\n        "time"\n    ],\n    "axis": "T",\n    "calendar": "proleptic_gregorian",\n    "long_name": "time",\n
"standard_name": "time",\n    "units": "hours since 1983-1-1 00:00:00"\n}''
   .Zmetadata for 'time' before : {'axis': 'T', 'calendar': 'proleptic_gregorian', 'long_name': 'time', 'standard_name': 'time', 'units': 'hours since 1983-1-1 00:00:00'}
   .Zmetadata for 'time' after : {'axis': 'T', 'calendar': 'proleptic_gregorian', 'long_name': 'time', 'standard_name': 'time', 'units': 'hours since 1983-1-1 00:00:00', '_ARRAY_DIMENSIONS': ['time']}
> Updated Array<zarr.attrs.Attributes object at 0x7f2eaf506190>
   Key : Value
   ['axis: T', 'calendar: proleptic_gregorian', 'long_name: time', 'standard_name: time', 'units: hours since 1983-1-1 00:00:00', "_ARRAY_DIMENSIONS: ['time']"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment