Good discussion of the limits of reinforcement learning from human feedback (RLHF).