20262 entries

  1. Temperature does one thing, and it is not creativity

    It is a single division applied before softmax. Knowing that tells you which value to use and, more usefully, what the value cannot buy you.

  2. ChatGPT does not know. It predicts.

    There is no fact table inside a language model. There is a function that guesses the next small piece of text, run over and over, and that turns out to be enough.