Descriptions:
Dylan Davis, who runs an AI consultancy and uses Claude, ChatGPT, and Gemini daily across client engagements, makes the case that newer, smarter models have quietly become worse at admitting uncertainty — a phenomenon he calls the ‘honesty gap.’ He cites an OpenAI research paper on the topic and pairs it with the concept of automation bias: as AI sounds more confident, users check its outputs less, compounding errors over time.
The bulk of the video is a practical walkthrough of three prompt rules designed to close that gap. The first forces blank-field outputs with required explanations when the model is uncertain, grounding responses strictly to source documents. The second recalibrates the model’s incentive structure by explicitly stating that a wrong answer carries three times the cost of a blank answer. The third adds a self-audit step requiring the model to flag its own assumptions and inferences separately from extracted facts.
Davis demonstrates each rule against a real contract-extraction scenario, showing side-by-side outputs with and without the prompts. The before-after comparison reveals that without the rules, AI fills in ambiguous fields like payment terms (where two conflicting clauses exist) by silently picking one; with the rules, it surfaces the conflict and leaves the decision to the user. The prompts themselves are short — a few lines each — making them immediately portable to any Claude, ChatGPT, or Gemini workflow.
📺 Source: Dylan Davis · Published March 28, 2026
🏷️ Format: Tutorial Demo
