LLMs can't synthesize
And no one's dealing with it
I was looking into Yoji Akao's book on Quality Function Deployment last week.
First thing I noticed: half the pages are charts.
Guess what it takes for an LLM to convey the same level of information as a single one of these charts?
I ran the numbers. For one A3-size chart, the LLM wrote down:
8 sections
33 paragraphs
1,048 words
7 A4 pages for the same level of information
A person can scan the chart in about 30 seconds and start digging into what matters. You can kickstart a team discussion around that chart in a matter of minutes.
Meanwhile, the LLM-produced document requires the entire team to go through the 7 pages and mentally map the related information before anything.
I draw to explain concepts, problems, solutions to my team almost every day. A freeform sketch settles an argument faster than any documentation I've written. I'll go further: knowing how to draw is a core engineering skill, and I think we should teach it the way we teach coding.
It's also exactly what LLMs can't do. Ask one for a diagram and you'll see it. It produces a picture of a chart, not a chart that does the thinking for you.
So here's what I keep coming back to. We've taught these models to write, to code, to reason out loud. We haven't taught them to draw, to take a mess of relationships and compress it into one thing you can see.
Until we do, "synthesis" stays a human skill. And I'm not sure the labs are even pointing at it.
Be honest. When did an LLM last hand you a diagram you could actually use without redrawing it?