When moving the patches generated by tf.extract_volume_patches or tf.image.extract_patches back to its original shape with tf.reshape or np.reshape methods, the result will be incorrect.
Above issue is explained in below extensively.
Note Stop right here if just need non overlapping patches:
- see if
np.reshapefits your bill - which usually does, then use that! (np-extract-patches.ipynb)
https://stackoverflow.com/questions/31527755/extract-blocks-or-patches-from-numpy-array
-
also for 2d images, for non-overlapping patches, one can use
tf.nn.space_to_depthandtf.nn.depth_to_space. Seetf-space-to-depth.ipynb. -
alternatively if you prefer c++, git ITK a try.
https://discourse.itk.org/t/patch-extraction/3630
-
for 2d or 3d images, and overlapping patches, one could reorder the pixels/voxels, see
tf-extract-patches-index.ipynb -
another solution by
Marco Anconais "to realize that the inverse of tf.extract_image_patches is its gradient" provided in below link. (see `tf-extract-patches-gradient.ipynb) https://gist.github.com/aewhite/14db960f9e832bce4041bf185cdc9615 -
I have not tried this one yet - there is a pytorch solution by
blanNLhttps://stackoverflow.com/questions/68150248/how-to-extract-overlapping-patches-from-a-3d-volume-and-recreate-the-input-shape
docker run -it -p 8888:8888 -w /workdir -v $PWD:/workdir tensorflow/tensorflow:2.6.1-gpu-jupyter bash
bash prep.sh
jupyter notebook --ip=* --allow-root
whatever solutions you end up using, doesn't really matter during training.
You will for sure get a performance hit during inference. For problems dealing with 3d patches, tf.extract_volume_patches may be one method, but for medical images, depending on hardware, OOM may occur.
-
thus now you are back to for loops with python/numpy for preprocessing -> batch wise inference -> reshape
-
alternatively, chop up the volume to cubes (faster than for loops) -> feed that asyncronously to multiple serving end points -> gather the results and do the reshape.
-
wait until you get the budget to run jumbo gpus for inference 24/7.
https://stackoverflow.com/questions/41564321/split-image-tensor-into-small-patches
https://stackoverflow.com/questions/44047753/reconstructing-an-image-after-using-extract-image-patches
https://stackoverflow.com/questions/40731433/understanding-tf-extract-image-patches-for-extracting-patches-from-an-image
https://stackoverflow.com/questions/53451728/n-dimensional-sliding-window-operation-in-python-using-gpu-accelerated-libraries
https://stackoverflow.com/questions/35691947/3d-sliding-window-operation-in-theano
vae patch
Discovering Digital Tumor Signatures—Using Latent Code Representations to Manipulate and Classify Liver Lesions
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8269051
pediatric-ct-seg
https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=89096588#89096588bcab02c187174a288dbcbf95d26179e8
Uh oh!
There was an error while loading. Please reload this page.