• @Pieisawesome
    link
    English
    222 hours ago

    It’s because LLMs don’t work with letters. They work with tokens that are converted to vectors.

    They literally don’t see the word “strawberry” in order to count the letters.

    Splitting the letter probably separates them into individual tokens