Task Category | Task Prompt | Description | Inputs | Outputs |
---|---|---|---|---|
Image Captioning | <CAPTION> |
Generate a basic caption for an image | Image | Text |
<DETAILED_CAPTION> |
Generate a detailed caption for an image | Image | Text | |
<MORE_DETAILED_CAPTION> |
Generate a very detailed caption for an image | Image | Text | |
<REGION_TO_CATEGORY> |
Generate a category label for a specified bounding box | Image, Bounding box | Text | |
<REGION_TO_DESCRIPTION> |
Generate a description for a specified bounding box | Image, Bounding box | Text | |
Object Detection | <OD> |
Detect objects and generate bounding boxes with labels | Image | Bounding boxes, Text |
<DENSE_REGION_CAPTION> |
Detect objects and generate bounding boxes with their captions | Image | Bounding boxes, Text | |
<CAPTION_TO_PHRASE_GROUNDING> |
Detect and ground phrases from the caption with bounding boxes | Image, Text | Bounding boxes, Text | |
<OPEN_VOCABULARY_DETECTION> |
Detect objects based on provided text (open vocabulary) | Image, Text | Bounding boxes, Text | |
<REGION_PROPOSAL> |
Propose regions of interest with bounding boxes | Image | Bounding boxes | |
Segmentation | <REFERRING_EXPRESSION_SEGMENTATION> |
Generate a segmentation polygon based on text description | Image, Text | Polygon |
<REGION_TO_SEGMENTATION> |
Generate a segmentation polygon for a given bounding box | Image, Bounding box | Polygon | |
OCR | <OCR> |
Extract text from the entire image | Image | Text |
<OCR_WITH_REGION> |
Extract text with location (bounding boxes or quadrilateral boxes) | Image | Text, Bounding boxes |
Created
October 9, 2024 13:28
-
-
Save Lihi-Gur-Arie/96a2b940da787f1951c16e02e7292690 to your computer and use it in GitHub Desktop.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment