Skip to content

Instantly share code, notes, and snippets.

@masadcv
Created May 26, 2021 11:31
Show Gist options
  • Save masadcv/84f1bc9f505056ea8f4290d14a002d2a to your computer and use it in GitHub Desktop.
Save masadcv/84f1bc9f505056ea8f4290d14a002d2a to your computer and use it in GitHub Desktop.
# Install required packages:
# !pip -q install simplecrf nibabel wget
# !BUILD_MONAI=1 pip -q install git+https://github.com/Project-MONAI/MONAI#egg=monai
from monai.networks.blocks import CRF
import wget
import nibabel
import denseCRF3D
import numpy as np
import torch
from functools import wraps
from time import time
# define a simple timing decorator, use for both simplecrf's densecrf3d and monai's CRF
def timing(f):
@wraps(f)
def wrap(*args, **kw):
ts = time()
result = f(*args, **kw)
te = time()
print('func:%r took: %2.4f sec' % (f.__name__, te-ts))
return result
return wrap
# run functions to get timings on the two functions
# densecrf3d run with timing
@timing
def densecrf3d(I, P, param):
return denseCRF3D.densecrf3d(I, P, param)
# MONAI's CRF run with timing
@timing
def runmonaicrf(I, P, crf_func):
return crf_func(P, I)
def load_sample_data():
# download sample data used in this example from github simplecrf repository
wget.download('https://github.com/HiLab-git/SimpleCRF/raw/master/data/2013_12_1_img.nii.gz')
wget.download('https://github.com/HiLab-git/SimpleCRF/raw/master/data/2013_12_1_init.nii.gz')
# Load some sample data
I1Nii = nibabel.load('2013_12_1_img.nii.gz')
PNii = nibabel.load('2013_12_1_init.nii.gz')
I1 = I1Nii.get_data()
P = PNii.get_data()
# convert input to intenstiy range of [0, 255]
I = np.asarray([I1], np.float32)
I = np.transpose(I, [1, 2, 3, 0])
I = I / I.max()* 255
I = np.asarray(I, np.uint8)
# probability map for each class
P = 0.5 + (P - 0.5) * 0.8
P = np.asarray([1.0 - P, P], np.float32)
P = np.transpose(P, [1, 2, 3, 0])
return I, P
if __name__ == '__main__':
I, P = load_sample_data()
print(I.shape)
print(P.shape)
# define same number of iterations to run for each method
ITERATIONS = 20
# run both methods on cpu
# 1. SimpleCRF's CRF with ITERATIONS
dense_crf_param = {}
dense_crf_param['MaxIterations'] = ITERATIONS
dense_crf_param['PosW'] = 3
dense_crf_param['PosRStd'] = 1
dense_crf_param['PosCStd'] = 1
dense_crf_param['PosZStd'] = 1
dense_crf_param['BilateralW'] = 5
dense_crf_param['BilateralRStd'] = 5
dense_crf_param['BilateralCStd'] = 5
dense_crf_param['BilateralZStd'] = 5
dense_crf_param['ModalityNum'] = 1
dense_crf_param['BilateralModsStds'] = (5,)
lab = densecrf3d(I, P, dense_crf_param)
# 2. MONAI's CRF with ITERATIONS - CPU
device='cpu'
I_pt = np.moveaxis(I, source=-1, destination=0)
P_pt = np.moveaxis(P, source=-1, destination=0)
I_pt = np.expand_dims(I_pt, axis=0)
P_pt = np.expand_dims(P_pt, axis=0)
I_pt = torch.from_numpy(I_pt).to(dtype=torch.float, device=torch.device(device))
P_pt = torch.from_numpy(P_pt).to(dtype=torch.float, device=torch.device(device))
crf_func = CRF(
bilateral_weight=5.0,
gaussian_weight=3.0,
bilateral_spatial_sigma=1.0,
bilateral_color_sigma=5.0,
gaussian_spatial_sigma=0.5,
update_factor=5.0,
compatibility_kernel_range=1,
iterations=ITERATIONS,
)
runmonaicrf(I_pt, P_pt, crf_func)
## output of the script on my local machine:
# func:'densecrf3d' took: 5.8184 sec
# func:'runmonaicrf' took: 177.5296 sec
@antoniocandito
Copy link

hello, thanks for sharing this script.
I am getting an error when I try to compute MONAI CRF...here the error

OptionalImportError: import monai._C (No module named 'monai._C').

I have installed MONAI in all possible way but still doesn't work...any advise please?

@masadcv
Copy link
Author

masadcv commented Sep 9, 2022

Hello @antoniocandito

You see this error as you are not installing MONAI with C++ extensions.

Please install with BUILD_MONAI=1 option:
BUILD_MONAI=1 pip install git+https://github.com/Project-MONAI/MONAI#egg=monai

More details here: https://docs.monai.io/en/stable/installation.html#option-1-as-a-part-of-your-system-wide-module

Additional context:
MONAI does not currently (as of 9th September 2022) ship with pre-compiled C++ binaries. If you wish to use C++/CUDA extensions in MONAI you have to install from source using BUILD_MONAI=1 environment variable.

@antoniocandito
Copy link

Hello Muhammad,
thank you very much for your help!

I could get running monai CRF, as well.
Any explanation why monai CRF takes longer to run?

I would like to train a model for semantic segmentation using as last layer the CRF (make sure that the network is aware of the CRF),
Instead of using the CRF as a post processing step.

Is there any tutorial in monai that I can refer to? or from other sources?

Thanks in advance for your help,
Antonio

@masadcv
Copy link
Author

masadcv commented Sep 10, 2022

Hi @antoniocandito
I am unsure why it is slower. I have a ticket on this which is still unaddressed for the most part. You may find the comments in that issue useful:
Project-MONAI/MONAI#2250

What is your use case? Are you looking at multi-class segmentation? Is your data 3D volume?
I believe for training, it may be helpful to look at CRF as RNN paper (https://www.robots.ox.ac.uk/~szheng/papers/CRFasRNN.pdf), which also has implementations in PyTorch (https://github.com/sadeepj/crfasrnn_pytorch). This seems to be implemented for only 2D images I believe.

AFAIK, SimpleCRF does not provide learnable CRF. While MONAI provides this, it may need to be verified with proper testing to understand how well it trains.

@antoniocandito
Copy link

Hi Muhammad,
I am looking for binary segmentation of data 3D volume.
You are right, MONAI provides learnable CRF, however, I cannot see any tutorial regarding the implementation and testing.
Have you seen any GitHub regarding this topic?

Thanks for your help,
Antonio

@masadcv
Copy link
Author

masadcv commented Sep 13, 2022

No I dont know about any tutorials on that, unfortunately.

In the absense of such tutorials, the best place to start is to look at the unittests for usage examples: https://github.com/Project-MONAI/MONAI/blob/8700fee0ececdf35e5b77fa097863928a4c74361/tests/test_crf_cuda.py#L508

I also have some transforms in MONAI Label that use CRF, you can check them out here:
https://github.com/Project-MONAI/MONAILabel/blob/1d6fefb482cc74ce5cc6330ff05bbe0cdc51fe01/monailabel/scribbles/transforms.py#L490

@antoniocandito
Copy link

Great, thanks sharing these links.
I can see that monai CRF has been used as post processing layer so far. It might be enough for improving my segmentations:)

Thanks for your help,
Antonio

@masadcv
Copy link
Author

masadcv commented Sep 13, 2022

Great, happy to help! You can easily extend these examples to train using the output of CRF. Just think of it as a layer, similar to convolution layer for example.

If all you need is post processing and you have binary segmentations, then CRF may be overkill.
You may want to try GraphCut/Max-flow for such post processing, as it will be faster and potentially a bit more accurate.

SimpleCRF already has a max-flow implementation, however it is restricted license for research only applications.
Alternative max-flow implementations with more permissive license can be found in the following github repositories:
https://github.com/masadcv/torchmaxflow
https://github.com/masadcv/numpymaxflow

@antoniocandito
Copy link

great, thanks a lot!.
I will do some testing and keep you posted with some progress, hopefully.

I might have a last (naive) question, in case of false negative segmentations, simpleCRF or GraphCut/Max-flow might help or not?

Thanks for your help,
Antonio

@masadcv
Copy link
Author

masadcv commented Sep 13, 2022

It totally depends on how complex and difficult a given case is. If it is a much simpler segmentation problem, then post-processing can definitely help recover from missed segmentations.

If you have further questions, I recommend sending me a message on MONAI slack channel: https://github.com/Project-MONAI/MONAI#community - I will be happy to help you more with this!

@ikatechis
Copy link

Hello and thank you for the script!

I am currently running a 33-label semantic segmentation with a UNet on CT scans and I was wondering how a CRF layer could improve me segmentation.

It is not clear to me however where I should put this layer. Should it be placed before or after the last convolution layer? Also I don't understand what the reference tensor should be for the MONAI implementation.

Thank you again,
Iasonas

@masadcv
Copy link
Author

masadcv commented Sep 15, 2022

Hi @ikatechis

You will need the output of your network after softmax (i.e. probabilites of 33 labels) as input of CRF. Regarding whether it will help, you need to try it out and look into literature for best configuration to use for such CT scans. These configurations are in the form of hyper-paramters for CRF layer itself. I recommend you read the CRF paper linked above in comments to understand this.

CRF is usually used as a post processing layer. This means that it post-processes the output of a network/model. With that said, there is nothing stopping you from using it within the network or separate from a CNN. Its just a matter of figuring out where it would help the most.

Hope that helps clear things for you!

@masadcv
Copy link
Author

masadcv commented Sep 15, 2022

If you have further questions, I recommend sending me a message on MONAI slack channel: https://github.com/Project-MONAI/MONAI#community - I will be happy to help you more with this!

@anisaaaaaaaaaaaa
Copy link

how can i install this version please

BUILD_MONAI=1 pip install git+https://github.com/Project-MONAI/MONAI#egg=monai

@masadcv
Copy link
Author

masadcv commented Mar 8, 2023

Please see https://docs.monai.io/en/stable/installation.html#option-1-as-a-part-of-your-system-wide-module for details about installation.
You need to type the BUILD_MONAI=1 pip install git+https://github.com/Project-MONAI/MONAI#egg=monai in your terminal to build the CRF features in monai. More details about this are in the link I mentioned in this commnet.

@anisaaaaaaaaaaaa
Copy link

anisaaaaaaaaaaaa commented Mar 8, 2023

when i try that, i get an error :

C:\Users\Demo 1A\Documents>BUILD_MONAI=1 pip install --no-build-isolation git+https://github.com/Project-MONAI/MONAI#egg=monai
'BUILD_MONAI' n’est pas reconnu en tant que commande interne
ou externe, un programme exécutable ou un fichier de commandes.

Do you know why i get this error when using BUILD_MONAI=1 before pip ?

@masadcv
Copy link
Author

masadcv commented Mar 8, 2023

when i try that, i get an error :

C:\Users\Demo 1A\Documents>BUILD_MONAI=1 pip install --no-build-isolation git+https://github.com/Project-MONAI/MONAI#egg=monai
'BUILD_MONAI' n’est pas reconnu en tant que commande interne
ou externe, un programme exécutable ou un fichier de commandes.

Do you know why i get this error when using BUILD_MONAI=1 before pip ?

Are you using windows?
Then try the following from cm/powersheel:
set BUILD_MONAI=1
pip install --no-build-isolation git+https://github.com/Project-MONAI/MONAI#egg=monai

Note: you will need relevant compilation tools on your Windows machine as this is compiling the monai c++ code. If the above fails then try to use python setup and see what error it gives:
set BUILD_MONAI=1
git clone https://github.com/Project-MONAI/MONAI
cd MONAI
python setup.py install

@anisaaaaaaaaaaaa
Copy link

anisaaaaaaaaaaaa commented Mar 10, 2023

hello , i did run this on my virtuel env

set BUILD_MONAI=1
git clone https://github.com/Project-MONAI/MONAI
cd MONAI
python setup.py install

it looks like it managed to finish the processing of the dependencies

Finished processing dependencies for monai==1.1.0+105.g354c9c2b

however when i run my script to apply the CRF , i still get the same error :

monai.utils.module.OptionalImportError: import monai._C (No module named 'monai._C').

Do you have an idea why is that ?

@masadcv
Copy link
Author

masadcv commented Mar 10, 2023

Hi @anisaaaaaaaaaaaa ,
Thats good progress. I can see that you are able to run setup and from your previous comment it appears you are using powershell and that BUILD_MONAI was not detected by setup.py script.

I think I know the issue here. For compiling MONAI you need to set environment variable BUILD_MONAI=1. However, on windows there are two methods of setting this depending on whether you are using CMD or Powershell.

On CMD it should be as I told in my previous comment:

set BUILD_MONAI=1
git clone https://github.com/Project-MONAI/MONAI
cd MONAI
python setup.py install

On Powershell it should be the following (notice env variable is set differently here):

$env:BUILD_MONAI=1
git clone https://github.com/Project-MONAI/MONAI
cd MONAI
python setup.py install

Once you are able to run this command and no longer see the message:
"Please set environment variable BUILD_MONAI=1 to enable Cpp/CUDA extension build."

Then we can debug whether your machine has any issues compiling the CPP code. Until now, it has not attempted compiling CPP because BUILD_MONAI was not properly set.

Hope this helps.

@anisaaaaaaaaaaaa
Copy link

hello @masadcv , i managed to run this lines with powershell on windows:

set BUILD_MONAI=1
git clone https://github.com/Project-MONAI/MONAI
cd MONAI
python setup.py install

it looks like it managed to finish the processing of the dependencies

Finished processing dependencies for monai==1.1.0+105.g354c9c2b

however when i run my script to apply the CRF , i still get the same error :

monai.utils.module.OptionalImportError: import monai._C (No module named 'monai._C').

Do you have an idea why is that ?

@masadcv
Copy link
Author

masadcv commented Mar 10, 2023

For Powershell, you need to change set BUILD_MONAI=1 to $env:BUILD_MONAI=1 as I indicated in my last comment

Successful install does not mean it has compiled CPP code. Check install logs and you will see that it skipped compilation because BUILD_MONAI was not set using proper method for Powershell

@anisaaaaaaaaaaaa
Copy link

okey i see,

I stick with the cmd then ,

I just run the comand lines

set BUILD_MONAI=1
git clone https://github.com/Project-MONAI/MONAI
cd MONAI
python setup.py install

Now i get this error :

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\include\crt/host_config.h(160): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2019 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.2\\bin\\nvcc.exe' failed with exit code 2

Do you know why i get this one ?

@masadcv
Copy link
Author

masadcv commented Mar 11, 2023

Now i get this error :

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\include\crt/host_config.h(160): fatal error C1189: #error:  -- unsupported Microsoft Visual Studio version! Only the versions between 2017 and 2019 (inclusive) are supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.2\\bin\\nvcc.exe' failed with exit code 2

Do you know why i get this one ?

Hi @anisaaaaaaaaaaaa ,
Thanks for getting back with the error message you get.
This error means you have unsupported Microsoft Visual Studio version. As the error states, you need a version between 2017 and 2019 to compile this on your machine.

If you need more help on this, I recommend taking this conversation offline. As mentioned above, please try to reach me on Slack and I can help guide you.

If you want to fix this on your own, look at installing Microsoft Visual Studio 2017 or 2019 with C++ build tools.

@AnisBoubekeur
Copy link

hey @masadcv ,

Thanks you for your help.

It would be grreat , what is your slack ?

@masadcv
Copy link
Author

masadcv commented Mar 14, 2023

Hi @AnisBoubekeur
If you join MONAI slack using instructions from here: https://github.com/Project-MONAI/MONAI#community
You can send me message at: Muhammad Asad with the same profile picture.
I would be happy to assist you further with this setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment