Large language models, like Chat GPT, have a known sycophancy problem. What this means is that they are designed to be helpful, and to prioritize being helpful over other priorities, like being accurate. I tried to find out why this is the case, and it seems it is because they use Reinforcement Learning from Human
The post LLMs Will Lie to be Helpful first appeared on NeuroLogica Blog.
