• @BetaDoggo_
    link
    English
    11 year ago

    It’s not even close, less than half of 3.5’s 85.5% in ARC. Some larger Open models are competitive in Hellaswag, TruthfulQA and MMLU but ARC is still a major struggle for small models.

    3Bs are kind of pointless right now because the machines with processors capable of running them at a usable speed probably have enough memory to run a 7B anyway.