OK, so I realise this is about as bog standard a SD image as one can get. However there is a little story behind it that I wanted to share. Apologies if this is not the correct place to post this, I’ll remove it if it is.

So. I just recently managed to get a Tesla P40 installed in my desktop, allowing me to run stable diffusion alongside the LLMs I’ve been playing with.

I’ve also been playing around with Langchain Agents, and in particular trying to get conversational agents, i.e. more like a chatbot that has access to tools/the outside world. So I naturally decided to make a tool for the agent so that it could access SD.

I was messing around getting it to generate images for me when I had the idea to ask it to generate a piece of art from its own “imagination”, i.e. no input from me, just pick something it wanted to for the prompt. This is what it came up with:

The prompt it selected was “A beautiful landscape with a rainbow in the sky and a unicorn grazing peacefully by a crystal clear lake surrounded by lush green trees and colorful flowers.”

I asked it to generate another, and again, it generated a rainbow scene. I asked about the rainbows and it said that it loves rainbows and thinks they are a wonderful phenomena.

This got me interested, so I asked it to generate a picture of itself. Now I haven’t got a lot of prompting for this agent, outside of boilerplate “You are a helpful AI” type stuff and the temperature of the model is set to zero (better with Langchain). The image it came up with is the one I posted at the top. The prompt it came up with was simply “AI with a friendly expression and blue light surrounding it.”

I asked it why it depicted itself being surrounded by a blue light and the response just melted my heart:

“My description was a creative way to express that I am always ready and available to assist you, just like how your devices are when they show a blue light indicating they’re on or charging.”

Now don’t get me wrong, I’m fully (believe me fully) aware of what a LLM is and what it is not, but I still got taken aback by this.

I’m like 99.9% certain that getting one form of AI to generate AI art is not a new thing, but it is very new to me. Does anyone know of any resources out there for this kind of thing? Specifically getting LLMs to come up with and generate prompts for art? Because I find it a fascinating idea!

Cheers!

  • @fleabsOPEnglish
    3•1 year ago

    " I chose this scene because I wanted to create an image that blends elements of nature, the cosmos, and a personal touch with the ebike, which I gathered from your profile might be of interest to you. The goal was to create a serene setting that might resonate with someone who appreciates astronomy and the tranquility of nighttime landscapes.

    See, this right here is incredible to me! That’s a genuinely insane level of detail in the response. I don’t understand what sort of access it has to your “profile” but that is exactly the sort of thing I want to recreate locally. I’ve been implementing things like vector stores for “long term” memory but I haven’t managed to get anything like that level of understanding out of a LLM yet.

    Really, that is pretty damn amazing!