Two Examples of Features – from OpenAI & Anthropic – Which Mitigate Bad Prompts

Rodin's The Gates of Hell, via https://en.wikipedia.org/wiki/File:Le_penseur_de_la_Porte_de_lEnfer_(musée_Rodin)_(4528252054).jpg)

Among new or novice prompters, prompts are generally too short. They lack context, rarely feature examples, and provide few descriptions. Chatbot makers and model providers have come to understand the tyranny of the black text box and adopted measures to help their users.

One of my favorite measures is how ChatGPT rewrites bad image-generation prompts before it hands them to DALL-E. Back in November of 2023, I wrote:

You don’t have to remember the incantations and phrase books needed to get good results. You don’t have to hang out on the Stable Diffusion Discord and make notes of keywords strings to conjure the results you may want. DALL-E in ChatGPT is better for nearly everyone than DALL-E, Stable Diffusion, or Midjourney on their own.

Revisiting this feature, I decided to poke behind the curtains and see how the prompts are being modified1.

Visiting ChatGPT, I prompted it with the terrible prompt, “Create an image of a statue thinking.”

Behind the scenes, ChatGPT rephrased that as:

A detailed statue of a human figure in a thinking pose, similar to Rodin’s ‘The Thinker.’ The statue is made of weathered stone, sitting on a pedestal, with a contemplative expression. The setting a peaceful outdoor environment, with soft lighting and a slightly cloudy sky.

Quite a difference! We can see the details from the modified prompt, plain as day:

The image DALL-E created from the above prompt.

But this raises a question: do all our image prompts get rephrased in ChatGPT?

There are plenty of scenarios where someone wouldn’t want their prompt edited, but there’s already plenty of toggles and drop-downs in ChatGPT’s interface. Adding another (“Don’t modify this prompt”) wouldn’t just add clutter, but needlessly confuse users unaware of this feature.

Thankfully, OpenAI has an elegant solution.

Before we reveal the trick, a brief detour…

Bad prompting isn’t limited to end users. People building applications hitting LLM APIs commonly write short prompts, that lack detail, which results in unsatisfactory and inconsistent results. More than a few times, I’ve chatted with frustrated developers, who think LLMs are over-hyped because a bad prompt couldn’t enable a simple feature. The problem, every time, is their prompt.

And the tool I pointed each of them to is truly a hidden gem: Anthropic’s prompt generator.

Log into their API console and hit the “Generate a prompt” button. Describe the task you want an LLM to complete, hit “Generate”, and dig into your new, effective prompt.

Anthropic's prompt generator, given our input.

Let’s give the prompt generator the following input, inspired by ChatGPT’s DALL-E prompt improvement feature:

A user will provide a prompt to generate an image from an image model. This prompt will likely be too short, lack description, and will generate a boring or unintended output. Given this input prompt, generate a better prompt that is more descriptive and likely to generate a more polished image. Provide plenty of details and clearly describe the image you wish to create based on the prompt.

Anthropic’s tool generated a much improved 236-word prompt – 3.5x longer than our input! It set the context, specified an output format, enumerated specific guidelines, spurred the LLM to reason first, and reiterated the goal simply.

Populating this prompt with our, “Create an image of a statue thinking,” prompt, we get some reasoning followed by this enhanced prompt:

A weathered bronze statue of a philosopher in a contemplative pose, seated on a stone pedestal with one hand supporting its chin, situated in an overgrown classical garden at dusk. The sculpture’s patinated surface catches the warm golden light of the setting sun, creating dramatic shadows that accentuate its pensive expression and furrowed brow. Small vines and moss partially climb the base of the monument, while a shallow reflecting pool in front mirrors the statue’s thoughtful silhouette in photorealistic detail.

Fantastic. Anthropic’s prompt generator truly is an incredible tool. For each of the developers who reached out to me, one try at the prompt generator and some light editing completely fixed their features.

The prompt generator should be the first stop for any developer, with little AI experience, looking to ship an LLM-powered function.

Armed with our Anthropic-generated enhanced prompt, we can digress back to ChatGPT and its DALL-E prompt editing.

If we drop our enhanced prompt into ChatGPT and hit enter…it doesn’t edit our prompt.

The app recognizes this prompt as sufficiently detailed and passes it along – unaltered – to DALL-E. This is excellent UX, a perfect example of a quiet AI feature: if you give ChatGPT a bad image prompt, it’ll improve it; but if you give it a good one, it won’t interfere.

The image DALL-E created from the prompt we created in Anthropic.

  1. If you want to follow along, jump into the developer tools of your web browser and hang out in the “Network” tab. There you can observe the prompt modifications as they’re passed around as JSON payloads. ↩