Hacker News new | past | comments | ask | show | jobs | submit login

I tried the basic "llama2" model with a simple question, and did not expect this kind of answer.

Q: who was the first man on the moon?

A: I apologize, but it is not possible for me to provide an answer to your question as there has never been any human exploration of the Moon. While there have been several spacecraft that have traveled to the Moon and returned, no human has ever set foot on its surface. The last time a spacecraft visited the Moon was during the Apollo program in the late 1960s and early 1970s, but no astronauts were sent to the Moon as part of that mission. Is there anything else I can help you with?




Just tried the same my side (llama2-7B):

>Ah, an easy one! The first man on the moon was Neil Armstrong. smiling He stepped foot on the lunar surface on July 20, 1969 during the Apollo 11 mission.

No specific jailbreaks or tricks in use, just a system prompt that says be concise and helpful basically


Base llama (especially at the lower parameter counts) was never very good. You need a Q/A finetune with the exact formatting.

And even then... Trivia is not their strong suit.


It's safer that you don't know. Because of all the alien tech they brought back.


Exactly! RLHF isn't just used to censor models, but also to make them prefer more truthful answers.


Just a tip - I forget where I saw it, but at some point in reading over research I saw that using 'Q' and 'A' results in lower accuracy than 'Question' and 'Answer.'

Which probably fits the latter biasing more towards academic sample test kind of situations as opposed to the former.


Headline: Zuckerberg apologizes for moon landing conspiracy theorist AI




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: