FastGPT can’t answer some questions about famous Chinese locations, history, or cultural events

atdrendel

Steps to reproduce:
Ask either of the following questions:

“Explain the history, significance, and important things to visit at 丽山园, near Xi’an in China. Answer in English, and translate your answer into easy-level Simplified Chinese.”

“Describe what happens in 贵妃醉酒 and the significance of it. Provide a list of ten key words and expressions in Simplified Chinese and their meanings.“

You’ll receive an answer saying something to the effect “I do not have enough information to…”.

Expected behavior:
I’d expect FastGPT to search for the specified terms and present an answer summarizing the places or events mentioned. Both of the examples I provided are quite well-known in the Chinese-speaking world, and there are lot of references to them on the web.

Debug info:
Running FastGPT on my iPhone in Safari connected to ASIA-EAST. I also connected my U.S. Central VPN, and the responses seemed the same.

Image/Video:

Comment
I’m not sure if this deserves to be its own bug since FastGPT is still a “Lab” project, but I figured you might not know about this shortcoming. So, I decided to post it in case it could prove helpful.

Vlad

atdrendel FastGPT is optimised for speed, not for depth of answers or accuracy. Answering this in a way you want would take much more than 900ms goal that we set for FastGPT and is a goal of another labs project, ExpertGPT, which we are currently working on.

atdrendel

Makes sense. Thanks for the reply.

Vlad

atdrendel How do you like this?

atdrendel

Vlad I'd grade the answer as "fair". The answer is a good summation of the overall story of Yang Guifei, but "The Drunken Beauty" is a Beijing Opera talking about a night when the Emperor visited another concubine instead of Yang Guifei, which made her angry. This, in turn, prompted her to get drunk.

The play doesn't tell the larger story of Yang Guifei, but ExpertGPT's answer makes me think it does, which is why I'd rate the answer as "fair".

The Chinese terms are correct and seem well-chosen, but, again, they are chosen from Yang Guifei's longer story, not from the Beijing Opera the question asked about.

Still, it's better than the answer I received from GPT 3.5, which was pretty much entirely wrong, as far as I can tell:

"贵妃醉酒" (Guìfēi zuì jiǔ), also known as "The Drunken Concubine," is a traditional Chinese folk song that tells the story of Yang Guifei, the beloved concubine of Emperor Xuanzong during the Tang Dynasty. In the song, Yang Guifei drinks too much wine and starts to feel homesick, causing her to break down in tears.

As she cries and sings, she remembers her hometown in Xi'an and the sweet memories of her childhood. The Emperor hears her sobbing and tries to comfort her, but to no avail. At the end of the song, Yang Guifei is still crying and singing, and the Emperor laments the fact that he cannot cure her homesickness.

10 key words and expressions in Simplified Chinese and their meanings:

贵妃 (guìfēi) - "Concubine;" a term used to refer to a female consort of a monarch or nobleman in ancient China.

醉酒 (zuì jiǔ) - "Drunkenness;" the state of being intoxicated with alcohol.

怀念 (huáiniàn) - "Homesick;" to feel nostalgic or homesick for a place or time from the past.

咽泪 (yàn lèi) - "Sobbing;" to cry quietly or with difficulty.

唱歌 (chànggē) - "Singing;" to sing or perform music.

陕西 (Shǎnxī) - "Shaanxi;" a province in central China, where Xi'an is located.

故乡 (gùxiāng) - "Hometown;" the place where someone was born or grew up.

美好 (měihǎo) - "Beautiful;" pleasant, enjoyable, or attractive.

心痛 (xīntòng) - "Heartache;" emotional or physical pain that is felt in the heart.

安慰 (ānwèi) - "Comfort;" to console, soothe, or reassure someone who is upset or distressed.