And if so, how do they label headphones, contact lenses etc?

  • @nandeEbisu
    link
    11 year ago

    They don’t process words as unified tokens for something like an LLM, but they do process them as multi-letter encoding, like byte-pair encoding or more advanced techniques.