In short: research reported by IEEE Spectrum found that AI models can now write better prompts for themselves than people can. I think the next step is AI tools that interview the user about the outcome they want and build the prompt for them. If that happens, the skills that matter most at work move away from talking to the technology, and towards clear thinking, subject knowledge and judgement.
What is prompt engineering?
Prompt engineering is the practice of writing and refining the instructions given to a generative AI model, such as a large language model, to get the best possible output. Not that long ago, it seemed to be all anyone could talk about. There were courses on it, job adverts for it, and plenty of people calling it the skill of the future.
I recently read an article in IEEE Spectrum by Dina Genkina called “AI Prompt Engineering Is Dead”, and I thought it was worth sharing, because it suggests things have already moved on.
Can AI write better prompts than people?
According to the research in the article, in many cases it can. Rick Battle and Teja Gollapudi at VMware tested 60 combinations of prompts across three open-source AI models on maths problems. The prompts the models generated for themselves beat the best human-written prompts in almost every case, and took hours to find rather than days. The researchers also found that popular techniques, such as asking the model to reason step by step, helped in some cases and hurt in others, with no reliable pattern.
Intel Labs found something similar with image generation. Its NeuroPrompts tool turned simple requests into detailed prompts that outscored those written by human experts.
This approach is known as automated prompt optimisation: using AI to search for and refine the prompt, rather than relying on a person to get it right by trial and error.
Does that mean prompt engineering is dead?
I don’t think the skill disappears, but I do think it changes shape. The article makes the point that building a prototype is easy, but turning it into a reliable product is much harder. Austin Henley, formerly of Microsoft, put it simply: “It’s very easy to make a prototype. It’s very hard to production-ize it.” Getting AI into real use needs reliability, security, testing and compliance, and a good prompt on its own doesn’t deliver those. The article suggests the work is being absorbed into broader roles, such as LLMOps (large language model operations) engineering.
What will replace the prompt box?
This is where I think it gets interesting. If the models are already better at writing prompts than we are, why are we still asking people to learn how to write them?
I think the answer is in the interface. Rather than the user having to craft the perfect prompt, the AI tool could do what a good consultant does at the start of a project, and interview you. What are you trying to achieve? Who is it for? What does a good outcome look like? After two or three rounds of questions back and forth, the tool builds the detailed prompt for you behind the scenes.
Some tools are already starting to ask a clarifying question or two before they get going, so I don’t think we’re far off. If we get it right, it removes the idea that you need a technology expert sitting between you and the AI to get real value from it. You just need to be clear about what you want.
How will AI change the role of the workplace employee?
If that’s where we’re heading, the ask of the people using these tools changes quite a lot. The technical side becomes less important, and the human side becomes more important.
People will need to be really clear about the outcome they want, and able to explain it well when the tool starts asking questions. They’ll need the experience to know what good looks like, so they can spot when something is wrong rather than just accepting it. And they’ll need to be comfortable owning the result, because however the work was produced, it’s still their name on it.
So I think the day job moves from doing the work to directing and checking it.
What skills should we teach the workforce of tomorrow?
That makes me wonder whether we’re focusing our training in the right place. Rather than teaching people how to talk to the technology, I think we should be investing in the skills that have always mattered:
- Clear thinking: being able to define a problem and the outcome you want before you start.
- Asking good questions: both of the AI and of the answers it gives back.
- Deep subject knowledge: the experience to know what good looks like in your field.
- Judgement: the confidence to challenge an answer that doesn’t feel right, and to take ownership of the final result.
None of these are new. But as the technology gets easier to use, they’re likely to become the main thing that separates good work from average work.
Frequently asked questions
Understanding how prompts work is still useful, particularly for people building AI products. For most employees, though, I think the more valuable skills are defining outcomes clearly and judging the quality of what comes back.
It’s the use of AI to generate, test and refine prompts automatically, rather than a person writing them by trial and error. The VMware research found it beat human-written prompts in almost every case it tested.
LLMOps stands for large language model operations. It covers the work needed to take an AI model into a reliable product, including prompt design, testing, monitoring, safety and compliance.
Source
Genkina, D. (2024) “AI Prompt Engineering Is Dead”, IEEE Spectrum, 6 March 2024. https://spectrum.ieee.org/prompt-engineering-is-dead
About the author
Steve Denby is a co-founder and Senior Partner at AssetMax.ai, a technology intelligence and value creation partner to private equity funds, portfolio companies, sell-side advisors and family offices.
