error

2020-02-26 22:14:30.649866: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Tesla K80, Compute Capability 3.7
  0%|                                                   | 0/599 [00:00<?, ?it/s]2020-02-26 22:14:31.678574: W tensorflow/core/framework/op_kernel.cc:1610] Unknown: FileNotFoundError: [Errno 2] No such file or directory: b'/project/cq-instructors-1/project1/data/netcdf/GOES2016/GridSat-CONUS.goes13.2016.01.01.0130.v01.nc'
Traceback (most recent call last):

  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/script_ops.py", line 221, in __call__
    ret = func(*args)

  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 585, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "evaluator.py", line 203, in custom_data_generator
    station_dict = get_dictionary_by_station(dataframe, config, station, target_datetimes)

  File "evaluator.py", line 150, in get_dictionary_by_station
    test_df['img_seq'] = get_image_sequences_by_station_cached(df, config, station, index_intersec)

  File "evaluator.py", line 98, in get_image_sequences_by_station_cached
    crop_area=32)

  File "/project/cq-training-1/project1/submissions/team04/code/util/image_utils.py", line 62, in get_image_data
    with netCDF4.Dataset(filepath) as file:

  File "netCDF4/_netCDF4.pyx", line 2321, in netCDF4._netCDF4.Dataset.__init__

  File "netCDF4/_netCDF4.pyx", line 1885, in netCDF4._netCDF4._ensure_nc_success

FileNotFoundError: [Errno 2] No such file or directory: b'/project/cq-instructors-1/project1/data/netcdf/GOES2016/GridSat-CONUS.goes13.2016.01.01.0130.v01.nc'


2020-02-26 22:14:31.678897: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at iterator_ops.cc:929 : Unknown: FileNotFoundError: [Errno 2] No such file or directory: b'/project/cq-instructors-1/project1/data/netcdf/GOES2016/GridSat-CONUS.goes13.2016.01.01.0130.v01.nc'
Traceback (most recent call last):

  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/script_ops.py", line 221, in __call__
    ret = func(*args)

  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 585, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "evaluator.py", line 203, in custom_data_generator
    station_dict = get_dictionary_by_station(dataframe, config, station, target_datetimes)

  File "evaluator.py", line 150, in get_dictionary_by_station
    test_df['img_seq'] = get_image_sequences_by_station_cached(df, config, station, index_intersec)

  File "evaluator.py", line 98, in get_image_sequences_by_station_cached
    crop_area=32)

  File "/project/cq-training-1/project1/submissions/team04/code/util/image_utils.py", line 62, in get_image_data
    with netCDF4.Dataset(filepath) as file:

  File "netCDF4/_netCDF4.pyx", line 2321, in netCDF4._netCDF4.Dataset.__init__

  File "netCDF4/_netCDF4.pyx", line 1885, in netCDF4._netCDF4._ensure_nc_success

FileNotFoundError: [Errno 2] No such file or directory: b'/project/cq-instructors-1/project1/data/netcdf/GOES2016/GridSat-CONUS.goes13.2016.01.01.0130.v01.nc'


	 [[{{node PyFunc}}]]
  0%|                                                   | 0/599 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "evaluator.py", line 480, in <module>
    stats_output_path=args.stats_output_path,
  File "evaluator.py", line 430, in main
    target_time_offsets, dataframe, user_config)
  File "evaluator.py", line 337, in generate_all_predictions
    station_preds = generate_predictions(data_loader, model, pred_count=len(target_datetimes))
  File "evaluator.py", line 301, in generate_predictions
    for iter_idx, minibatch in enumerate(data_loader):
  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 622, in __next__
    return self.next()
  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 666, in next
    return self._next_internal()
  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/data/ops/iterator_ops.py", line 651, in _next_internal
    output_shapes=self._flat_output_shapes)
  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_dataset_ops.py", line 2673, in iterator_get_next_sync
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.UnknownError: FileNotFoundError: [Errno 2] No such file or directory: b'/project/cq-instructors-1/project1/data/netcdf/GOES2016/GridSat-CONUS.goes13.2016.01.01.0130.v01.nc'
Traceback (most recent call last):

  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/script_ops.py", line 221, in __call__
    ret = func(*args)

  File "/home/guest126/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 585, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))

  File "evaluator.py", line 203, in custom_data_generator
    station_dict = get_dictionary_by_station(dataframe, config, station, target_datetimes)

  File "evaluator.py", line 150, in get_dictionary_by_station
    test_df['img_seq'] = get_image_sequences_by_station_cached(df, config, station, index_intersec)

  File "evaluator.py", line 98, in get_image_sequences_by_station_cached
    crop_area=32)

  File "/project/cq-training-1/project1/submissions/team04/code/util/image_utils.py", line 62, in get_image_data
    with netCDF4.Dataset(filepath) as file:

  File "netCDF4/_netCDF4.pyx", line 2321, in netCDF4._netCDF4.Dataset.__init__

  File "netCDF4/_netCDF4.pyx", line 1885, in netCDF4._netCDF4._ensure_nc_success

FileNotFoundError: [Errno 2] No such file or directory: b'/project/cq-instructors-1/project1/data/netcdf/GOES2016/GridSat-CONUS.goes13.2016.01.01.0130.v01.nc'


	 [[{{node PyFunc}}]] [Op:IteratorGetNextSync]
2020-02-26 22:14:31.761380: W tensorflow/core/kernels/data/generator_dataset_op.cc:102] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
	 [[{{node PyFunc}}]]
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:Unresolved object in checkpoint: (root).encoder.cell.kernel
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).encoder.cell.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).encoder.cell.recurrent_kernel
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).encoder.cell.recurrent_kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).encoder.cell.bias
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).encoder.cell.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.cell.kernel
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).decoder.cell.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.cell.recurrent_kernel
2020-02-26 22:14:31,853 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).decoder.cell.recurrent_kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).decoder.cell.bias
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).decoder.cell.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).dense.layer-0.kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).dense.layer-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).dense.layer-0.bias
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).dense.layer-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).dense.layer-2.kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).dense.layer-2.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).dense.layer-2.bias
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).dense.layer-2.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).td.layer.layer-0.kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).td.layer.layer-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).td.layer.layer-0.bias
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).td.layer.layer-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).td.layer.layer-3.kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).td.layer.layer-3.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).td.layer.layer-3.bias
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).td.layer.layer-3.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.cell.kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.cell.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.cell.recurrent_kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.cell.recurrent_kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.cell.bias
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).encoder.cell.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.cell.kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.cell.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.cell.recurrent_kernel
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.cell.recurrent_kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.cell.bias
2020-02-26 22:14:31,854 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).decoder.cell.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-0.kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-0.bias
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-2.kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-2.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-2.bias
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).dense.layer-2.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-0.kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-0.bias
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-3.kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-3.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-3.bias
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).td.layer.layer-3.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.cell.kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.cell.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.cell.recurrent_kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.cell.recurrent_kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.cell.bias
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).encoder.cell.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.cell.kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.cell.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.cell.recurrent_kernel
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.cell.recurrent_kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.cell.bias
2020-02-26 22:14:31,855 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).decoder.cell.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-0.kernel
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-0.bias
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-2.kernel
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-2.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-2.bias
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).dense.layer-2.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-0.kernel
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-0.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-0.bias
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-0.bias
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-3.kernel
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-3.kernel
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-3.bias
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).td.layer.layer-3.bias
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/alpha/guide/checkpoints#loading_mechanics for details.
2020-02-26 22:14:31,856 [MainThread  ] [WARNI]  A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/alpha/guide/checkpoints#loading_mechanics for details.
(base) (ift6759-env-01) [guest107@helk80n03 code]$ /home/guest123/ift6759-env-01^C
(base) (ift6759-env-01) [guest107@helk80n03 code]$ ls /project/cq-instructors-1/project1/data/netcdf/GOES2016/
ls: cannot access '/project/cq-instructors-1/project1/data/netcdf/GOES2016/': No such file or directory
(base) (ift6759-env-01) [guest107@helk80n03 code]$ ls /project/cq-instructors-1/project1/data/netcdf/GOES2016/
ls: cannot access '/project/cq-instructors-1/project1/data/netcdf/GOES2016/': No such file or directory
(base) (ift6759-env-01) [guest107@helk80n03 code]$ ls /project/cq-instructors-1/project1/data/