ahmetkca
/ gist:b944c80445635cc67664817e250a379d
Created
April 5, 2024 04:36
— forked from mzbac/gist:00ebe60bb36fa4d8f65509f8e47350d5
llava implementation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ```sh | |
| Initial Setup: | |
| +-------------------+ +---------------+ | |
| | Text Sequence | | Raw Images | | |
| | [T1, <IMG>, T2, | | [Image1, | | |
| | T3, <IMG>, T4] | | Image2] | | |
| +-------------------+ +---------------+ | |
| Step 1: Convert Text and <IMG> Tokens to Embeddings | |
| +---------------------------------------------------------+ |