Scientists at Microsoft Research and their colleagues are developing an artificial intelligence (AI) system that can tell stories based on photos.
The aim is not just to explain what items are in the picture, but also what appears to be happening and how it might potentially make a person feel, US-based website livescience.com quoted the researchers as saying.
For example, if a person is shown a picture of a man in a tuxedo and a woman in a long, white dress, instead of saying, "This is a bride and groom," he or she might say, "My friends got married. They look really happy; it was a beautiful wedding."
"The goal is to help give AIs more human-like intelligence, to help it understand things on a more abstract level and what it means to be fun or creepy or weird or interesting," Margaret Mitchell, study senior author said in a
"With our focus on storytelling, we hope to help AIs understand human concepts in a way that is very safe and beneficial for mankind, rather than teaching it how to beat mankind," Mitchell, who is also a computer scientist at Microsoft Research, added.
To build a visual storytelling system, the researchers used deep neural networks, computer systems that learn by example.
The researchers said although everything worked fine, the computerized storyteller needs a lot more tinkering.
"The automated evaluation is saying that it's doing as good or better than humans, but if you actually look at what's generated, it's much worse than humans," Mitchell noted.
"There's a lot the automated evaluation metrics are not capturing and there needs to be a lot more work on them. This work is a solid start, but it is just the beginning."
The scientists will detail their findings in San Diego at the annual meeting of the North American Chapter of the Association for Computational Linguistics later this month.
