Skip to content

Instantly share code, notes, and snippets.

@Lihi-Gur-Arie
Created October 9, 2024 13:28
Show Gist options
  • Save Lihi-Gur-Arie/96a2b940da787f1951c16e02e7292690 to your computer and use it in GitHub Desktop.
Save Lihi-Gur-Arie/96a2b940da787f1951c16e02e7292690 to your computer and use it in GitHub Desktop.
Task Category Task Prompt Description Inputs Outputs
Image Captioning <CAPTION> Generate a basic caption for an image Image Text
<DETAILED_CAPTION> Generate a detailed caption for an image Image Text
<MORE_DETAILED_CAPTION> Generate a very detailed caption for an image Image Text
<REGION_TO_CATEGORY> Generate a category label for a specified bounding box Image, Bounding box Text
<REGION_TO_DESCRIPTION> Generate a description for a specified bounding box Image, Bounding box Text
Object Detection <OD> Detect objects and generate bounding boxes with labels Image Bounding boxes, Text
<DENSE_REGION_CAPTION> Detect objects and generate bounding boxes with their captions Image Bounding boxes, Text
<CAPTION_TO_PHRASE_GROUNDING> Detect and ground phrases from the caption with bounding boxes Image, Text Bounding boxes, Text
<OPEN_VOCABULARY_DETECTION> Detect objects based on provided text (open vocabulary) Image, Text Bounding boxes, Text
<REGION_PROPOSAL> Propose regions of interest with bounding boxes Image Bounding boxes
Segmentation <REFERRING_EXPRESSION_SEGMENTATION> Generate a segmentation polygon based on text description Image, Text Polygon
<REGION_TO_SEGMENTATION> Generate a segmentation polygon for a given bounding box Image, Bounding box Polygon
OCR <OCR> Extract text from the entire image Image Text
<OCR_WITH_REGION> Extract text with location (bounding boxes or quadrilateral boxes) Image Text, Bounding boxes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment