Skip to content

Instantly share code, notes, and snippets.

@linminhtoo
Created August 22, 2023 09:53
Show Gist options
  • Save linminhtoo/0c83d9420ad3c2c4a5b297800ab50220 to your computer and use it in GitHub Desktop.
Save linminhtoo/0c83d9420ad3c2c4a5b297800ab50220 to your computer and use it in GitHub Desktop.
error when running colabfold_batch
2023-08-22 18:58:05,566 Running colabfold 1.3.0 (26de12d3afb5f85d49d0c7db1b9371f034388395)
2023-08-22 18:58:28,127 Unable to initialize backend 'tpu_driver': NOT_FOUND: Unable to find driver in registry given worker:
2023-08-22 18:58:28,382 Unable to initialize backend 'tpu': INVALID_ARGUMENT: TpuPlatform is not available.
2023-08-22 18:58:35,545 generated new fontManager
Traceback (most recent call last):
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/bin/colabfold_batch", line 8, in <module>
sys.exit(main())
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/colabfold/batch.py", line 1728, in main
run(
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/colabfold/batch.py", line 1210, in run
from colabfold.alphafold.models import load_models_and_params
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/colabfold/alphafold/models.py", line 7, in <module>
from alphafold.model import model, config, data
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/alphafold/model/model.py", line 20, in <module>
from alphafold.model import features
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/alphafold/model/features.py", line 19, in <module>
from alphafold.model.tf import input_pipeline
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/alphafold/model/tf/input_pipeline.py", line 17, in <module>
from alphafold.model.tf import data_transforms
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/alphafold/model/tf/data_transforms.py", line 18, in <module>
from alphafold.model.tf import shape_helpers
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/alphafold/model/tf/shape_helpers.py", line 16, in <module>
import tensorflow.compat.v1 as tf
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/__init__.py", line 41, in <module>
from tensorflow.python.tools import module_util as _module_util
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/python/__init__.py", line 41, in <module>
from tensorflow.python.eager import context
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/python/eager/context.py", line 33, in <module>
from tensorflow.core.framework import function_pb2
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
File "/g/data/if89/apps/colabfold_batch/1.4.0/python3/3.9.2/lib/python3.9/site-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
_descriptor.FieldDescriptor(
File "/home/005/ml0997/.local/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 561, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
======================================================================================
Resource Usage on 2023-08-22 18:58:57:
Job Id: 93195220.gadi-pbs
Project: kx58
Exit Status: 0
Service Units: 0.62
NCPUs Requested: 12 NCPUs Used: 12
CPU Time Used: 00:00:07
Memory Requested: 48.0GB Memory Used: 1.22GB
NGPUs Requested: 1 GPU Utilisation: 0%
GPU Memory Used: 308.0MB
Walltime requested: 12:00:00 Walltime Used: 00:01:02
JobFS requested: 1.0GB JobFS used: 0B
======================================================================================
@linminhtoo
Copy link
Author

#!/bin/bash

#PBS -P kx58
#PBS -q gpuvolta
#PBS -l ncpus=12
#PBS -l ngpus=1
#PBS -l mem=48GB
#PBS -l walltime=12:00:00
#PBS -l jobfs=1GB
#PBS -l wd
#PBS -l storage=gdata/if89+scratch/kx58
#PBS -M "[email protected]"
#PBS -m abe

module use /g/data/if89/apps/modulefiles
module load colabfold_batch/1.4.0

# see https://opus.nci.org.au/display/Help/Colabfold
DATE=$(date +%d_%m_%Y_%H_%M_%S)
log_dir="/scratch/kx58/ml0997/colabfold_logs"
mkdir -p "${log_dir}"

run_name="1uao_chignolin"
# naming is numerical, if 1 input FASTA to search = 1 file in out folder
a3m_input_path="/home/005/ml0997/colabfold/search_outputs/1uao_chignolin/0.a3m"
out_folder="/home/005/ml0997/colabfold/fold_outputs/1uao_chignolin/"
mkdir -p ${out_folder}

colabfold_batch --amber --templates --num-recycle 3 --use-gpu-relax \
    ${a3m_input_path} \
    ${out_folder} \
    2>&1 | tee "${log_dir}/${run_name}_fold_${DATE}.log"

@linminhtoo
Copy link
Author

#!/bin/bash

#PBS -P kx58
#PBS -q normal
#PBS -l ncpus=24
#PBS -l mem=100GB
#PBS -l walltime=3:00:00
#PBS -l jobfs=1GB
#PBS -l wd
#PBS -l storage=gdata/if89+scratch/kx58
#PBS -M "[email protected]"
#PBS -m abe

module use /g/data/if89/apps/modulefiles
module load colabfold_batch/1.4.0

# see https://opus.nci.org.au/display/Help/Colabfold
DATE=$(date +%d_%m_%Y_%H_%M_%S)
log_dir="/scratch/kx58/ml0997/colabfold_logs"
mkdir -p "${log_dir}"

# FYI: took 49 mins for chignolin, 12 CPUs = 41 SUs
# 24 CPUs = 41 mins, not much faster. better to use 12 CPUs.
run_name="1uao_chignolin_24cpus"
num_threads=24
in_path="/home/005/ml0997/colabfold/data/1uao_chignolin.fasta"
out_folder="/home/005/ml0997/colabfold/search_outputs/${run_name}/"
mkdir -p ${out_folder}

colabfold_search --db-load-mode 1 --threads ${num_threads} \
    ${in_path} \
    $COLABFOLDDIR/database \
    ${out_folder} \
    2>&1 | tee "${log_dir}/${run_name}_search_${DATE}.log"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment