Skip to content

Instantly share code, notes, and snippets.

@Pyr-000

Pyr-000/1_H14.md

Last active September 22, 2023 22:56
Show Gist options
  • Save Pyr-000/f69335314be3abc5660a55fdd0643125 to your computer and use it in GitHub Desktop.
Save Pyr-000/f69335314be3abc5660a55fdd0643125 to your computer and use it in GitHub Desktop.
Download ZIP
UMAP maps of internal CLIP text representations (direct viewable links)
Raw
1_H14.md

UMAP maps of internal OpenCLIP ViT-H/14 text representations. Colors are derived from k_means clustering on the unprocessed (1024-dimensional) CLIP feature tensors.

This variant of CLIP is used in StableDiffusion version 2.x. (https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K)

"by <artist>": https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_artists_by_map_cluster.html

"in the style of <artist>": https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_artists_st_map_cluster.html

adjectives: https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_adjectives_map_cluster.html

nouns: https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_nouns_map_cluster.html

verbs: https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_verbs_map_cluster.html

'adverbs': https://gist.githack.com/Pyr-000/90503e58688a7827278ea801096da933/raw/cd7f59e3d7d25800329ef0213d352e7e8ca8e8c9/OpenCLIP_ViT-H14_rev3_adverbs_map_cluster.html

Wordlists are unchanged from the previous (L/14) maps. Therefore: the noun/verb/adjective wordlists are imperfect, and will contain some incorrect words. This is especially true for the 'adverbs' list.

Raw
2_L14.md

UMAP maps of internal CLIP ViT-L/14 text representations. Colors are derived from k_means clustering on the unprocessed (768-dimensional) CLIP feature tensors.

(now includes a search bar, only lowercase text for searching and improved clustering!)

This variant of CLIP is used in StableDiffusion version 1.x, as well as many older CLIP-based projects. (https://huggingface.co/openai/clip-vit-large-patch14)

"by <artist>": https://gist.githack.com/Pyr-000/8bf4a50fae8f57e41a21b9739010de60/raw/4140e62ed6b2eb88e08be35066496773613e8b0e/2022821235113_artists_by_map_cluster.html

"in the style of <artist>": https://gist.githack.com/Pyr-000/8bf4a50fae8f57e41a21b9739010de60/raw/4140e62ed6b2eb88e08be35066496773613e8b0e/2022821235113_artists_st_map_cluster.html

adjectives: https://gist.githack.com/Pyr-000/8bf4a50fae8f57e41a21b9739010de60/raw/4140e62ed6b2eb88e08be35066496773613e8b0e/2022821235113_adjectives_map_cluster.html

nouns: https://gist.githack.com/Pyr-000/8bf4a50fae8f57e41a21b9739010de60/raw/4140e62ed6b2eb88e08be35066496773613e8b0e/2022821235113_nouns_map_cluster.html

verbs: https://gist.githack.com/Pyr-000/8bf4a50fae8f57e41a21b9739010de60/raw/4140e62ed6b2eb88e08be35066496773613e8b0e/2022821235113_verbs_map_cluster.html

'adverbs': https://gist.githack.com/Pyr-000/8bf4a50fae8f57e41a21b9739010de60/raw/4140e62ed6b2eb88e08be35066496773613e8b0e/2022821235113_adverbs_map_cluster.html

The noun/verb/adjective wordlists are imperfect, and will contain some incorrect words. This is especially true for the 'adverbs' list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment