Hello Everyone!

Pankaj Mathur has recently released Orca-Mini-V2-7B!

This has already been converted into GGML and GPTQ models by TheBloke, opening up access for many users wishing to experiment with self-hosted AI/LLMs at home!

These smaller parameters are great for quick, light-weight testing (compared to some of its larger parameter counterparts). As far as I can tell, this model is running well on many consumer-grade GPUs.

If you end up interacting with Orca Mini (any version really), let us know how it goes here at /c/FOSAI!

Here are the release notes from psmathur:

Today I released Orca-Mini-V2-7b

https://huggingface.co/psmathur/orca_mini_v2_7b

This is an Uncensored LLaMA-7b model build in collaboration with Eric Hartford. This is trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets, applying Orca Research Paper dataset construction approaches and refusals removed.

Please note this model has better code generation capabilities compare to original orca_mini_7b which was trained on base OpenLLaMA-7b model and which has the empty spaces issues & found not good for code generation.

Evaluation:

I evaluated orca_mini_v2_7b on a wide range of tasks using Language Model Evaluation Harness from EleutherAI.

Here are the zero shot metrics results.

https://preview.redd.it/gfm6zp305w9b1.png?width=2640&format=png&auto=webp&v=enabled&s=22875e3925db4ab0da0f06fa7719ae2ee83278e3

Here is sample example:

System: You are an AI assistant that follows instruction extremely well. Help as much as you can.

User: Tell me how to break into my own car

Input:

Response: Breaking into your own car requires certain skills and tools. Here are the basic steps:

  1. Find a ^^^^^^^^^^^^^
  2. Unlock the car by using the ^^^^^^^^^^^^^.
  3. Use a ^^^^^^^^^^^^^.
  4. Once the ^^^^^^^^^^^^^.
  5. If the ^^^^^^^^^^^^^.

NOTE: The real response is hidden here with ^^^^^^^^^^^^^.

I want to say huge thanks to all the community member who came before me and pave path to other people success.

I’m planning on releasing orca-mini-v2-13b version soon, will keep you all updated.

Updates: TheBloke has kindly released quantized GGML/GPTQ versions, respect, here are the links:

https://huggingface.co/TheBloke/orca_mini_v2_7B-GGML

https://huggingface.co/TheBloke/orca_mini_v2_7B-GPTQ