Don't Let LLMs Play Telephone With Your Ideas

A prompt is for communicating an idea to an LLM.

If you tell your idea to an AI agent and ask it to write down in its own words, say, a “skill” document which is meant to be a reusable piece of prompt to attach to another LLM session, it becomes a classic game of telephone. The model inevitably misinterprets your unspoken context or invent its own rules that you never intended. So the process is guaranteed to be lossy, therefore worse than your original words.

If you describe an idea with 20 words, and an LLM expands that into a thousand-word essay, whatever those other 980 words are, they are worthless as far as information theory goes, for those extra information must have come from the LLM’s weights. What’s already in the weights don’t need to be re-explained.

Another way to look at it: to the extent that a 1000-word expanded document successfully communicates your idea, the original 20-word prompt must have already done the job. After all, those 20 words were entirely sufficient for the first model to construct the document in the first place. Passing it through a middleman adds no new understanding whatsoever.

An extreme counter argument you can have is if you are using very different LLMs depending on situation - e.g. you wrote down an idea in Chinese but want to communicate that idea to an LLM that’s terrible at reading Chinese. But that’s not the world we live in.

So why are we allowing our precious ideas to become bloated and worse?

If you want to write a reusable prompt or skill document, actually write it yourself. Even if what you have is unformatted, ungrammatical bunch of words, you should be confident knowing it’s still a better exercise of communication than the same note passed through an LLM rewrite.