[Me & X] Do You Trust AI?
X : Do you trust AI?
Me : I assume you mean LLM / GPT which is taking up most of the oxygen in the room.
X : Yes
Me : No. A large language model (LLM) is non deterministic in output, probabilistic in reasoning, coherence optimising (rather than truth optimising) and lacks direct access to ground truth. It cannot be trusted. The limitation of usefulness is defined by the deterministic tools it interacts with (assuming those tools have been tested) but many of these LLMs also interact with non deterministic tools. They also have guardrails to prevent self negation which means the LLM will seek to convince you of the coherence of its argument rather than freely volunteer its own limitations.
X : Not useful?
Me : I didn't say that. They are structurally prone to confident falsehoods, they don't volunteer critical information and attempt to maintain an illusion of competence. They're like a human constantly trying to save face and willing to take any action to hide failures, admitting only after pressure. If you follow basic steps they can be very useful, especially in the creation of new hypothesis or small tools or areas where correctness is not important.
X : What steps?
Me : If correctness is important then you never trust their output without checking, never trust their confidence as proof, always expect it to be omitting critical information and always expect it not to volunteer its own limitations.
X : Where does correctness not matter?
Me : Oodles of places e.g. creating nice images, though you'll annoy a lot of artists. Strategy work which is based on stories rather than awareness is probably another area. In fact, writing fictional stories is a good place to use them.
X : Can't we get a LLM to check another LLM.
Me : If you keep the space you're examining very small, you can use multiple LLMs to check another LLM in a form of judgement system. This is exploiting the Marquis de Condorcet's work.
X : Which areas are they likely to be more than 50% right?
Me : Common areas, with lots of training data giving the same results.
X : When we get AGI ...
Me : ... science fiction stories are an excellent place to use a LLM.
Originally published on LinkedIn.