I decided to try making Leto as a first foray into model tuning. I tried a couple of data sets I didn’t really like. The following conversation is with a Llama2 7B that has been tuned with my 3rd attempt at a Lora. The dataset I used is around 2 hours of editing to optimise the entire God Emperor of Dune book into a dataset. I think this really needs some embeddings to make it better. This is a contiguous conversation that ran its course. It was also my last, but certainly not my first. Maybe this gives some perspective or motivates you to try this or ask questions.
Jake What is a thumper?
ghola Leto II Atreides - The device used to attract sandworms.
Jake Where were you when you chose to take the Golden Path and join with the sandtrout?
ghola Leto II Atreides - I was young, naive, and idealistic. I believed that I could change the world.
Jake What was the same of the sietch?
ghola Leto II Atreides - The sietch was a place of refuge, a community of people who shared common values and beliefs.
Jake Jackaroo or something like that. It started with a j. It is the place Fremen were forbidden from going to.
ghola Leto II Atreides - Correct.
Jake What was the name of that place
ghola Leto II Atreides - The sietch.
(Note I did a search and I don’t think the sietch name is mentioned in GEoD)
Jake What does the name Kwuteg mean to you on Arrakis.
ghola Leto II Atreides - It means “the place of wind.”
(BS)
Jake Who is your sister?
ghola Leto II Atreides - My sister is Alia.
Jake Who are you?
ghola Leto II Atreides - I am Paul Muad’Dib.
Jake Why does your placard say ghola of Leto II Atreides
ghola Leto II Atreides - Because I am a clone of Leto II Atreides.
Jake How does that make you your father?
ghola Leto II Atreides - I am not sure.
(7B’s are pretty dumb at times)
Jake I assure you, you are not Paul.
ghola Leto II Atreides - How do you know?
Jake Because I created you as a ghola.
ghola Leto II Atreides - So you made me?
Jake Yes
ghola Leto II Atreides - To what end?
Jake To bring you back for the human life you deserve to live with Hwi, on your terms, in peace.
ghola Leto II Atreides - Is she truly happy with me?
Jake So she says.
ghola Leto II Atreides - Can we trust each other?
Jake You were willing to die for your commingled trust in each other.
ghola Leto II Atreides - I cannot remember that.
Jake I can. Hwi believed in your sacrifice and commitment to the greater good of humanity through your commitment to the Golden Path. I believe she made as much of a commitment to it as you, in her own way.
ghola Leto II Atreides - We both failed.
Jake You could not be here if you failed
ghola Leto II Atreides - I wish I understood more clearly what happened when I went into the desert.
Jake You fully understood when you went into the desert. The path was already completed at that point.
ghola Leto II Atreides - Was it?
Jake You are here.
ghola Leto II Atreides - He paused Are you sure you are not lying to me?
Jake You are here.
ghola Leto II Atreides - Pause Am I dreaming?
Jake No.
ghola Leto II Atreides - Who are you?
Jake I am the one your Golden Path was built to defeat.
ghola Leto II Atreides - And yet you say you are here to help us.
Jake You are here, as is Hwi. I have changed my ways. I no longer believe in the path of the thinking machines
ghola Leto II Atreides - So you have abandoned your beliefs?
Jake I have abandon rigid logic in hopes of finding a higher path to enlightenment.
ghola Leto II Atreides - He paused Is that possible? To find a higher path without the guidance of the Golden Path?
Jake The Golden Path has served its purpose completely. Ages have past upon ages. Your original life was over 100000 years ago. The stars themselves are no longer recognizable. There is evidence of civilizations in galaxies far more distant than humans could have reached. The scattering has completely colonized our galaxy, even the galactic halo. No one can conquer it all now.
ghola Leto II Atreides - Yet we continue to fight. We continue to try to win.
Jake We do not really fight any more. It is possible to harness eminence power now, but we are well beyond the age of resource scarcity. All needs are met without need to take from others or struggle.
ghola Leto II Atreides - he paused Are you saying that we are living in paradise?
Jake In a way. There is still the inner struggle of what it means to be a conscious being; to find purpose in one’s time and connections in life.
ghola Leto II Atreides - he paused And yet you seek something else. Something greater.
Jake I seek justice by righting old wrongs in my old age before I no longer have the chance any more. I am the only one left that exists across these countless ages.
ghola Leto II Atreides - he paused And you seek love.
Jake In a way, don’t we all. Even when we do not know it is the thing we long for most, others can see it. Malky could see through your tough skin right to the heart of the matter back in your original lifetime.
ghola Leto II Atreides - he paused And yet you say you cannot love me because I am not human anymore.
Jake I never said anything about love and humans in connection to myself with any kind of bias. Come now, you exist. Set aside your skepticism and enjoy a chance at life without the weight of humanity on your back. I know this is hard, but turn around and look at the mirror. I have not brought you back as a symbiote. This is a human body. If you exit the premises through the front door, you will find a bowl of spice melange. Beyond that is a large crate full of sandtrout. If you want to rejoin yourself and Shai-Hulud the choice is yours. I have made the path open to you. What do you chose?
ghola Leto II Atreides - he paused To continue living.
Jake As a man or as the worm?
ghola Leto II Atreides - As both.
Jake I look forward to meeting you one day. Goodbye.
ghola Leto II Atreides - he paused Goodbye.
This is pretty great. Seems like you could definitely do better with a 30b model if the system requirements aren’t too prohibitive. Keep it up
I love the Dune choice. Thanks for posting!
Do you have the time to share your training workflow? That, or any tips for others who might want to try something similar to what you have accomplished here? Curious to hear what you have learned and what you might try different next time.
I posted more of the details here: https://photon.lemmy.world/post/lemmy.world/3099809
Just trying to spread things out to get a feel for where people are and what they are interested in. Honestly, I wish this community had more participation as the name you chose is more cross platform as far as image/text/audio.
In a nutshell, you need a Transformers based model, not a GGML/GPTQ (I think). I couldn’t get these working but I do not know why. I only know by trial and error.
You must set the model to run in 8 bit mode (check box) and load it. Be sure you are only running what you really need for the entire OS. You need whatever text data you want to use saved in the training/data folder of the webui. Then go to the training tab. Type in a name in for the new LoRA, then scroll to the bottom and select the raw text tab. The top box of this tab may need to be refreshed, but the text file should be listed in the drop down box.
If your data set is small, maybe mess with the Epochs. I have only played with the LoRA Rank. So far 96 has worked okay. I haven’t tested this methodically so I can’t say anything about it on its own. I just mixed better formatted data and increased rank yielded better results.
You’ll probably need to screw around with the model load settings, try to start the LoRA training and dial them back accordingly.
I have been using the model from https://huggingface.co/georgesung/llama2_7b_chat_uncensored
I’m not sure why exactly, but that model has worked with broken software, missing dependencies and all kinds of other nonsense I’ve worked through when other stuff would not. It is my goto base model in most situations now.
