Image Captioning

Image Captioning related modeling class

class pororo.tasks.image_captioning.PororoCaptionFactory(task: str, lang: str, model: Optional[str])

Bases: pororo.tasks.utils.base.PororoFactoryBase

Generates textual description of an image

English (transformer.base.en.caption)

• dataset: MS-COCO 2017 (Tsung-Yi Lin et al. 2014)

• metric: TBU

Examples

>>> caption = Pororo(task="caption", lang="en")
>>> caption("https://i.pinimg.com/originals/b9/de/80/b9de803706fb2f7365e06e688b7cc470.jpg")
'Two men sitting at a table with plates of food.'

static get_available_langs()

static get_available_models()

load(device: str)

Load user-selected task-specific model

Parameters

device (str) – device information

Returns

User-selected task-specific model

Return type

object

class pororo.tasks.image_captioning.PororoCaptionBrainCaption(extractor, generator, tokenizer, translator, device, config)

Bases: pororo.tasks.utils.base.PororoSimpleBase

predict(image: str, **kwargs)

Predict caption using image features

Parameters

image (str) – image path

Returns

generate captiong corresponding to input image

Return type

str