$ cd ./blog

Why I created null

From my original dictation to an edited draft and a revised article: why I created a skill that asks AI to wait while I think.

Three versions of the same article. Switch at any point to compare the corresponding passage. Tap an original passage, or choose “Compare this passage,” to see what changed.

Original: dictated words, with paragraph breaks. Edited: the first AI-shaped draft and my corrections. Revised: the article after source checks, critique, and several exchanges. These are three stages, not every intermediate draft.

Compare changes

Corresponding passages keep their place. Blank space marks a shorter version.

For roughly three months now, I’ve been heavily using AI agents. Before that, I used chats fairly regularly to create applications and small tools, copy-pasting back and forth between an IDE and Claude or ChatGPT. One of the things that came out of that progression is a skill called null. It asks the AI to reply “Received.” while I think out loud, then do the work when I specifically ask for something. I wanted to choose when it started contributing.

Then I started using agents in my homelab. Once I was able to implement things at a rapid pace, including building out my Kubernetes lab and migrating my services from Docker Swarm, something became apparent: those little ideas, or solutions to problems I was dealing with, were things I could turn into software with the agents available at the time. Not just bespoke code solutions to individual issues, but actual engineered solutions that were broader and applied to anything within the domain of the problem they were built to solve.

I started with my understanding of software engineering and how software functions, then used agents to research and establish best practices. That became my first AI harness, which was essentially a set of Copilot instructions, some of which applied when a file matched a particular pattern. Those instructions have since migrated into skills, including an engineering assurance skill for planning before substantial implementation and reviewing implemented work before release.

Slowly but surely, I’ve built a more comprehensive harness around the coding agents I use, namely Codex and Claude. If you look at my GitLab activity over the past several months that I’ve been using agents, it’s essentially a solid white box. I’ve been iterating, designing, and planning practically every little thing that pops into my head, and I’m getting close to releasing a couple of projects I’ve been working on for some time.

Working on those projects has also meant continually improving the harness. That’s become a project in its own right, one I hope to make public eventually, though I’m not ready to talk about it quite yet.

Part of creating what I consider an effective harness is keeping up with AI research and development, including the related social and psychological research. There are studies directly relevant to the design choices I’m making, and I think it’s important to cross-check myself and vet whatever hypotheses I come up with. Odds are I’m not the first person to have thought about something. There’s also a good chance that my explanation is way off base for some very good reasons.

Which brings me to an ongoing conversation with my wife.

She’s concerned about how much time I spend interfacing with AI. She has very good reasons to be concerned about how much time I spend on anything. Say the words “Dota 2,” as in Valve’s online MOBA, then step back and watch her reaction.

Don’t worry, I won’t leave you hanging. I have about 5,000 hours in that game.

But I digress.

Our conversations forced me to consider more deeply what effect AI is having on me. Being the person I am, I felt I had some well-thought-out arguments for why the time and effort I’ve put into working with and understanding these systems wasn’t the same problem we’ve all seen discussed online and in news articles. I’m talking about things described as “AI psychosis,” along with deskilling and offloading cognition to the machine. Those questions grew out of our conversation, but they don’t settle her concern about the time itself. Null doesn’t answer that either.

Most of my argument has been that how someone engages with AI matters. I’ve written about AI amplifying what was already there before. I think AI is a multiplier. I was also making an argument about myself: I thought that being willing to question my own ideas and check what the model told me would help protect against the problems my wife was worried about. That was my reasoning, but it isn’t evidence that I’m protected. The studies discussed here don’t establish a protective personality profile or tell me what is happening to me. They also don’t establish a cause of psychosis.

I should also define what I mean by AI, because that word seems to mean something different to everyone. Here, I’m talking about large language models. In my fairly informal description, they’re incredibly good guessing machines built out of the magic of statistics. More precisely, models of this kind learn patterns during training and generate text by predicting tokens, which can be words or parts of words, conditioned on the text they’re given. That distinction between training and the current prompt matters. Brown et al., 2020.

After our most recent conversation, I decided to do another pass through the research, specifically looking at how AI might harm human cognition. Does it? To what degree? What does the science actually say?

What I found was: it depends.

Go figure.

I had read the papers. For this version of the article, I asked AI to check the studies I was referring to and fill in the references. We then went back and forth over the draft, including the difference between my original claims and the qualifications it had added.

The math study involved nearly 1,000 high-school students. Both GPT-4 tools helped them perform better during practice. But when the AI was taken away, the group using the less constrained tool scored 17% below the control group. The tool built around teachers’ tutoring guidance largely avoided that penalty, though it didn’t produce a statistically significant improvement on the unaided exam. Students using the less constrained tool also didn’t recognize how much worse their learning and performance had been. That’s a short-term learning result; it doesn’t tell me what months of using coding agents have done to me. Bastani et al., 2025.

The colonoscopy study was a retrospective before-and-after comparison at four Polish centres. Across 1,443 procedures performed without AI, the proportion detecting at least one adenoma fell from 28.4% before AI was introduced to 22.4% afterward: six percentage points. That raises a concern about deskilling, but this observational comparison cannot establish that AI caused the decline. It also studied a medical detection tool, not a conversational language model. Budzyń et al., 2025.

Granted, colonoscopies are a slightly different use case from what I assume the average person is doing with AI. lol.

Over the next couple of days, I kept thinking about what I’d read and began incorporating that research into how my harness functions. At one point, as I was getting into the shower, I had a thought.

I’ve spent a lot of time teaching people, especially at work, how to use AI effectively. Even during the chat phase of my own use, one of the things I talked about was what I called “context poisoning”: the information and framing you give a model can affect what comes out. I’m using that phrase informally here. A prompt influencing a response is normal model behavior; it doesn’t require malicious input or mean that every part of a prompt changes every answer.

Sure. Duh.

But you aren’t the only one supplying input. There’s the system prompt, any instructions or skills the system loads, and whatever else the product puts behind the scenes that we don’t necessarily see. Not to be conspiratorial about it. And beyond the immediate context, there’s the training data that shaped the model in the first place.

All of that affects the output. The way you ask a question can steer the response toward solution A rather than solution B. It might never mention solution B at all. If you didn’t know solution B existed, you’ve unwittingly missed the opportunity to learn about it. There’s a related, studied failure mode here: sycophancy. Sharma and colleagues found that assistants sometimes favored answers agreeing with a user’s stated beliefs over truthful ones. Sharma et al., ICLR 2024.

I realize that’s always a possibility, but finding unknown unknowns is one of the things I find most useful about AI. I’ll often give it a vague question with instructions to search widely, including things that might not initially seem related. I want that information so I can judge it for myself, because these models aren’t something I’m willing to trust at face value.

That brought me back to my wife’s concerns.

These models are trained on human statements, thoughts, and words. Their responses can reproduce ways that humans communicate and influence one another. If we can “poison” a model’s context, could it do something similar to ours?

Could it affect what I’m considering, what I’m overlooking, and how I’m arriving at a decision?

That prompted another round of research. The human “context window” is my analogy, not a claim that human memory works like an LLM. Two relevant papers address different parts of that question.

Glickman and Sharot studied 1,401 participants across experiments involving perceptual, emotional, and social judgments. Biased systems increased bias in people’s judgments, and participants underestimated some of that influence. Accurate systems could also improve independent judgments. These experiments used several kinds of systems, including hard-coded algorithms and image generation; they weren’t a test of extended conversations with coding agents. Glickman and Sharot, published online in 2024, journal issue 2025.

For conversational language models, Salvi and colleagues ran a randomized study with 900 participants in short debates. GPT-4 given personal information was more persuasive than the human baseline. However, a September 2026 correction says its advantage over GPT-4 without personal information was not statistically significant. It also doesn’t tell us what happens over months of ordinary agent use. Salvi et al., 2025; 2026 correction.

That’s what led me to attempt an admittedly ham-fisted solution: the null skill.

My thinking was that if the AI stopped volunteering things I hadn’t asked for, maybe I could limit how much it influenced my own decision-making context. The skill’s instructions ask it to respond to the operation I actually requested, while leaving the things I hadn’t supplied unresolved. While I’m thinking aloud, it should say “Received.” An explicit request permits the requested work, then it returns to quiet behavior.

Those are instructions, not a guarantee. Null doesn’t erase earlier context, remove the effects of training, override higher-priority instructions, or make the model neutral. An outline or critique I ask for can still change the direction I take. Staying quiet could also mean it withholds a suggestion I would have found useful. I still want that help when I ask for it. Whether choosing when it contributes reduces unwanted influence remains a hypothesis.

I dictated the original version of this article into a Codex session in VS Code using Handy, with null mode active, on September 23. It was my first real use of null. My impression was that it forced me to think through what I was doing. Later in the same conversation, I turned null off and asked for a review. I could decide when I wanted the model’s opinion, even though that doesn’t tell me how much it influenced me.

That’s a personal observation from one attempt. The repository’s testing note records 20 written behavioral checks, nine of which were run once each against an earlier version. Those check instruction-following, not cognitive outcomes. A few synthetic tests, some anecdotal evidence, and a selection of research papers do not turn my idea into an accepted theory. I don’t know whether this will make a meaningful difference beyond what I’ve noticed while using it to dictate this article.

That uncertainty is part of why I published the skill and wanted to write about it. It’s also why I want to share this on every social service I have the stomach to use.

X.com can kiss my ass.

I’m hoping it reaches enough people to hear how it works for them. I also hope it encourages people to take the time to think and reason through things they care about.

The studies are linked above so you can read them for yourself. You can find null and its instructions on GitHub. Give it a try if you’re interested. Tell me what you were trying to do and whether the waiting helped or just got in the way.

The revised article’s AI-use note follows the comparison.

The documents

Read or save each version as Markdown, including the comparison with exact text changes.

What changed between versions

Null article: comparison of the three versions

Updated 2026-09-24. These are three stages of the article: the original dictation, the first edited draft with Max’s corrections, and a revised article developed through source checks, critique, and several exchanges. Max confirmed that he had read the papers. The first two documents are unchanged. The third incorporates the editing direction he accepted after asking for a critique; its new prose was written by Codex. Accepting an editing direction does not make the assistant’s additions independently authored by Max. Intermediate versions are retained in project history rather than added as more reader tabs.

Scroll sideways to see all columns.

Version File What it contains
1 Original dictation Max’s original dictation, with paragraph breaks only. Original grammar, spelling, punctuation, and uncertainty remain.
2 Edited draft The assistant’s first draft, with Max’s two subsequent corrections. It already includes paraphrases, reordered text, softer claims, and new connective wording.
3 Revised article Version 2 developed through source checks, research additions, critique and subsequent editing, with the AI-use note describing that process.

The two exact diffs at the end compare 1 → 2 and 2 → 3. They expose the differences between these retained stages; the table below supplies the three-way reading guide. They do not treat version 2 as human-only writing.

Three-way comparison

Scroll sideways to see all columns.

Passage Original dictation (V1) Edited draft (V2) Revised article (V3) Origin of the change
Opening History of using AI agents. Condensed that history. Adds an early description of null’s requested behavior. Assistant wording following requested critique
Software scope Ideas could be accomplished with agents. Max’s subsequent correction: software addressing the broader problem domain. Retained unchanged. Author correction
GitLab activity Solid white box, without an explicit period in that sentence. Max’s subsequent correction: the past several months of agent use. Retained unchanged as a personal report, not an independently audited activity metric. Author correction
Engineering assurance Post-design, pre-implementation, post-implementation modes. Rephrased the same three-part description. Aligned with the current prebuild/prerelease phases. Assistant factual correction
Wife, time spent, Dota 2 Personal account, including 5,000 hours. Condensed repetitions; removed “While I love my wife, obviously”. Removes the aside about technical knowledge; explicitly separates time spent from the influence null attempts to limit. No new claim about the relationship. Assistant revision following critique
AI as multiplier and protective habits Broad claims about laziness, impressionability, and self-critical people. Changed categorical claims into “my argument”, “could”, and “might”; moved the contrasting passage together. Preserves the account of Max’s reasoning, removes the lazy-versus-deep-thinker contrast, and states that it is not evidence of personal protection. Assistant changes to certainty and framing, followed by author-requested revision
Math research Unspecified study, major skill loss, overestimated capability. Short generalized account plus a research placeholder. Named and sourced study, measured endpoint, and limits; see research changes below. Assistant research and correction
Colonoscopy research Said a study showed a similar effect. Softened to a concern about skill loss. Named and sourced study, measured endpoint, and observational limit. Assistant research and correction
Models, training, context Guessing-machine shorthand; training discussed with other input. Separated training from immediate context. Added a primary model reference and qualified the informal “context poisoning” label. Assistant technical clarification
Solutions A and B Input can affect which possibilities the user sees. Tightened wording; preserved the example. Preserved it and added a related sycophancy example, explicitly traceable below. Assistant supporting connection
Human context analogy Models could “poison the human context window”; research unspecified. Recast as questions and left a placeholder; “Could it affect…” is assistant-written phrasing. Kept the question, marked the analogy, and supplied two bounded research examples. Assistant research and framing clarification
Null behavior Conditional hypothesis: answering only requests might limit influence. Described withholding unsolicited material; added “Maybe that would give me more room to think.” Grounds requested behavior in the public skill; adds that requested output can influence the user and silence can withhold useful suggestions. Removes the invented “more room” thought. Assistant critique and revision; hypothesis retained
First real use “Claude Astra eye” session; first use since yesterday; personal impression. Changed ambiguous identity to AI and “absolutely forces” to a personal report; added “I have to work through what I want to say.” Identifies Codex, dates the account to September 23, removes the incorrect yesterday claim and invented “work through” sentence, and records the later explicit request for critique after null was turned off. Session record and assistant revision; experience remains Max’s account
Synthetic tests and effectiveness A few tests, anecdote, selected papers do not establish theory. Added an explicit statement of uncertainty beyond this dictation. Linked the repository testing note and distinguished behavioral checks from cognitive outcomes. Assistant clarification and sourced testing scope
Publication and outreach Skill public; wording also sounds as though article/social publication already happened. Softened publication wording; kept the X.com sentence. Public skill link verified; article sharing stated as intent. Assistant state correction
Reader feedback Hopes for additional data points. Tightened wording. Asks when the waiting helped or got in the way, without claiming those reports establish effectiveness. Assistant suggestion from requested critique
Sources and research summaries Promised links, possibly summaries. Added link placeholders. Replaced placeholders with public sources; did not publish or link private summaries. Assistant source work
Process note No separate disclosure paragraph. Assistant added a disclosure. Updates it for the several exchanges, source checks, requested critique and further revision; also identifies Codex’s comparison and page implementation. Assistant addition

Research and factual additions in version 3

Max confirmed during this revision: “I did read the papers”. His reading is therefore part of the author-supplied record. These explanations identify the assistant’s editorial additions and evidentiary checks without recasting his research as work the assistant originated. The exact added prose is in the V2 → V3 diff.

  1. Engineering-assurance terminology. Technical correction from the current skill: its phases are prebuild and prerelease; audit, remediation planning, authorized remediation, and reverification are its modes. This does not verify the historical migration of every instruction.

  2. Earlier blog reference. Added a verified public cross-link. The earlier post discusses amplification of laziness in publishing; it does not establish the broader psychological claim.

  3. Personal protection remains a hypothesis. Assistant-added qualification, not a newly reported belief or experience from Max. The cited studies do not test a protective personality profile or this individual. The article does not use learning or persuasion studies as evidence about psychosis.

  4. What a language model predicts. Assistant-added technical explanation. Training is not literally text sitting in the current context window; prompting and changing model weights are different operations. Brown et al. provide a primary example, not documentation of every current product.

  5. Research added for this version. Max confirmed during this revision that he read the papers. The added sentence distinguishes that reading from the assistant’s subsequent reference checking and prose work.

  6. Math and colonoscopy evidence. Replaced the broad claim with measured outcomes and explicit limits. Primary paper text and the clinical abstract were checked; this is a factual revision, not only copy-editing.

  7. Context-poisoning terminology. Assistant-added technical qualification. Preserves Max’s label while avoiding inevitability or treating ordinary conditioning as a demonstrated attack.

  8. Concrete evidence for user-input effects. Assistant-added supporting example and connection. Sycophancy is related to the source dictation’s concern, but Max did not name this study in that dictation. It does not prove the particular omitted-option scenario.

  9. Human-context analogy and influence evidence. Assistant-written summaries and source checks fill the explicit gap; Max confirmed that he read the papers. Added the beneficial direction of feedback and the boundary between non-LLM tasks and conversational persuasion. Checked the September 3, 2026 correction; the direct personalization comparison is p=0.0678, conventionally rounded to 0.07. Neither paper tests null or Max’s experience.

  10. What null requests and cannot guarantee. Checked against the public SKILL.md and README. Corrected a possible guarantee into requested behavior and preserved the untested hypothesis. These qualifications were added by the assistant from the implementation, not attributed to a new personal realization.

  11. Session identity. The original says “Claude Astra eye”; draft.md generalized that to AI. The dictation was made in a Codex session. No exact underlying model or effort is claimed; VS Code and Handy remain Max’s supplied account.

  12. Test evidence and anecdote. Replaced unspecified test evidence with the public repository’s reported scope. No tests were rerun here, no success rate was inferred, and no reduction in influence is claimed.

  13. Publication tense. The skill was already public when this article was first drafted. Article-sharing language was changed to intent instead of implying distribution had already occurred.

  14. Replace remaining link placeholders. Added the verified public project link and removed the placeholder. The research dossier is private; its existence does not supply a public research-summary URL. No private workspace or tracker links were put into the article.

  15. Writing-process disclosure. The third version is now labeled Revised article. Its note describes the back-and-forth process, Max’s corrections and request for another editing pass, and Codex’s prose and implementation. Agreement on an editing direction is not described as final approval of every sentence.

  16. Follow-up critique and chronology. The original’s “last night” and “yesterday” were not dependable chronology. Both are removed from the revised article without inventing a replacement shower date. The revised account records the September 23 first use and later request for critique from this conversation. The author accepted another editing pass after the assistant recommended distinguishing time spent from influence, avoiding a protective-personality claim, introducing null earlier, and asking for concrete usability feedback. These are assistant-originated editorial changes, not new autobiographical discoveries.

Source checks

Primary sources were checked on 2026-09-23. This was a targeted fact-check of this article, not a new systematic review. Earlier research notes supplied candidate references; the cited findings were checked against current primary pages or original paper text. No private dossier content was published.

Scroll sideways to see all columns.

Source Material inspected and use
Bastani et al., PNAS, 2025 Original PDF, pp. 1–4: trial design, outcomes, student perceptions. Correction inspected: author affiliation only.
Budzyń et al., 2025 Original indexed abstract: design, denominators, detection rates, interpretation. Full clinical article was not needed for the bounded description and is not claimed read.
Glickman and Sharot Publisher text: abstract, accurate/biased algorithm conditions, participant awareness, methods. Online 18 December 2024; journal issue 2025.
Salvi et al., 2025 Publisher text: experimental design, aggregate results, discussion. 3 September 2026 correction also inspected.
Sharma et al., ICLR 2024 Author research summary at Anthropic, posted 2023; narrow model-behavior claim only.
Brown et al., 2020 Original paper’s abstract and model description support the training/prompt distinction; not a live product-capability claim.
Public null README and SKILL.md Fetched current raw main files. Source documents establish requested behavior and reported test scope, not demonstrated cognitive benefit.
Written by no one Public article fetched; supports the earlier-blog cross-reference as a related argument, not scientific evidence.

What remains unknown or personal

  • Whether null reduces unwanted influence, improves thinking, or changes longer-term outcomes is unmeasured. No cited study tests null.
  • Max’s time using agents, Dota 2 hours, conversations with his wife, GitLab activity description, implementation history, and first-use experience remain his account. They were not independently audited or inferred from unrelated records.
  • Max confirmed that he read the papers; the research summaries and qualifications in version 3 were written by the assistant.
  • “AI psychosis” remains a report of a term Max has seen discussed. This draft does not establish a diagnosis, clinical causation, or a protective personality profile.
  • The first two versions retain the dictated relative dates as historical text. The revised article removes them; the exact shower date remains unestablished.
  • The harness was Codex. An exact live model identifier and reasoning setting were not independently established.
  • The private research summaries have no verified public URL in this work. Version 3 links public originals instead of suggesting the private summaries were released.

Exact text changes

These diffs compare the original dictation with the first edited draft, then that draft with the current revised article. They show changes between the three retained stages, not every intermediate editing exchange. Draft front matter is omitted. Each long line is one paragraph; a minus line is removed text and a plus line is added text.

Original dictation → edited draft
--- original-dictation.md
+++ draft.md
@@ -1 +1 @@
-I'll start from the beginning. For roughly three months now, I have been heavily using AI agents. Before that, I was using chats fairly regularly. I was creating applications or small tools using chat and just copy-pasting back and forth between an IDE and Claude or Chat GPT.
+For roughly three months now, I’ve been heavily using AI agents. Before that, I used chats fairly regularly to create applications and small tools, copy-pasting back and forth between an IDE and Claude or ChatGPT.
@@ -3 +3 @@
-And then I started using the agents in my home lab and once I was able to begin implementing things at a rapid pace, like my Kubernetes lab and migrating all of my services rapidly from Docker Swarm to Kate's, it became apparent to me that these little ideas that I would have tangentially for applications or solutions to problems that I had were certainly able to be accomplished effectively with the state of AI agents as it was at the time.
+Then I started using agents in my homelab. Once I was able to implement things at a rapid pace, including building out my Kubernetes lab and migrating my services from Docker Swarm, something became apparent: those little ideas, or solutions to problems I was dealing with, were things I could turn into software with the agents available at the time. Not just bespoke code solutions to individual issues, but actual engineered solutions that were broader and applied to anything within the domain of the problem they were built to solve.
@@ -5 +5 @@
-I started with my understanding of software engineering and how software functions, and began using agents to research and establish best practices and create what you would call my first AI harness, which was essentially just a set of co-pilot instructions with some that would apply on a glob match. Those same instructions have been migrated into skills, like an engineering assurance skill that has different modes for post-design, pre-implementation, and post-implementation.
+I started with my understanding of software engineering and how software functions, then used agents to research and establish best practices. That became my first AI harness, which was essentially a set of Copilot instructions, some of which applied when a file matched a particular pattern. Those instructions have since migrated into skills, including an engineering assurance skill with different modes for reviewing work after design, before implementation, and after implementation.
@@ -7 +7 @@
-And I've slowly but surely begun to build a more comprehensive and feature-rich harness around the coding agents that I use, namely Codex and Claude. If you look at my GitLab activity, you'll see that it's essentially a solid white box. I have been iterating and designing and planning every little thing that pops into my head, and I'm at the point where I'm very close to releasing a couple of projects that I've been working on for some time now.
+Slowly but surely, I’ve built a more comprehensive harness around the coding agents I use, namely Codex and Claude. If you look at my GitLab activity over the past several months that I’ve been using agents, it’s essentially a solid white box. I’ve been iterating, designing, and planning practically every little thing that pops into my head, and I’m getting close to releasing a couple of projects I’ve been working on for some time.
@@ -9 +9 @@
-Through my experience working on all of these projects, I've continued to improve and iterate the harness. And that in and of itself is a project that I hope to bring out publicly at some point, though I'm not ready to talk about it quite yet.
+Working on those projects has also meant continually improving the harness. That’s become a project in its own right, one I hope to make public eventually, though I’m not ready to talk about it quite yet.
@@ -11 +11 @@
-Part of creating what to me is an effective harness is ensuring that I not only stay up to date with the frontiers of artificial intelligence research and development But also the related social and psychological research. There are a number of studies that are applicable directly to the design choices that I am making I feel that it's important to cross-check myself and thoroughly vet any hypotheses that I come up with. Odds are I'm not the only one who's had a thought like X before, and odds are even greater that the thought that I had about X is way off base for some very good reasons
+Part of creating what I consider an effective harness is keeping up with AI research and development, including the related social and psychological research. There are studies directly relevant to the design choices I’m making, and I think it’s important to cross-check myself and vet whatever hypotheses I come up with. Odds are I’m not the first person to have thought about something. There’s also a good chance that my explanation is way off base for some very good reasons.
@@ -13 +13 @@
-Now, that said, I recently came upon some interesting research in relation to an ongoing conversation that my wife and I have been having. My wife is concerned. She's concerned about the amount of time that I'm spending interfacing with AI. While I love my wife, obviously, she's not exactly a technical person. And she also has very good reasons for being concerned about the amount of time that I'm spending on anything. Say the word, the words Dota two, as in valves online MOBA game and then step back and watch her reaction. Don't worry, I won't leave you hanging. I have about 5,000 hours in that game.
+Which brings me to an ongoing conversation with my wife.
@@ -15 +15 @@
-But I digress. Our ongoing conversation around the amount of time I've been spending using AI systems forced me to deeply consider the effect that AI is having on me and being the person that I am, I felt that I had some well-thought-out arguments as to why, in my case, the amount of time and effort I've been using the amount of time and effort that I have put into working with and understanding AI systems is not the same problem that I'm sure most people have seen posted about online or in news articles. Specifically, I'm talking about things like AI psychosis and problems like de-skilling, offloading cognition to the machine
+She’s concerned about how much time I spend interfacing with AI. My wife isn’t particularly technical, but she also has very good reasons to be concerned about how much time I spend on anything. Say the words “Dota 2,” as in Valve’s online MOBA, then step back and watch her reaction.
@@ -17 +17 @@
-Most of my arguments to my wife have been essentially that in those situations where somebody is using AI to a fault, it has a lot to do with the way that they engage with it. I've written about this on this blog before, but I will say it again: AI is a multiplier. If somebody was lazy or influenciable, impressionable before AI, then AI only accentuates those traits and causes additional issues as we've seen.
+Don’t worry, I won’t leave you hanging. I have about 5,000 hours in that game.
@@ -19 +19 @@
-I also think it's important here to define what I mean when I say AI because it seems to mean a different thing to different people. What I'm talking about is large language models that are basically incredibly good guessing machines built out of the magic of statistics
+But I digress.
@@ -21 +21 @@
-On the other hand, if you are already a deep thinker, self-critical in a positive way, in a productive way, then you likely already have a lot of the traits and built-in self-checks that help to prevent the kinds of issues that I that my wife is worried about.
+Our conversations forced me to consider more deeply what effect AI is having on me. Being the person I am, I felt I had some well-thought-out arguments for why the time and effort I’ve put into working with and understanding these systems wasn’t the same problem we’ve all seen discussed online and in news articles. I’m talking about things described as “AI psychosis,” along with deskilling and offloading cognition to the machine.
@@ -23 +23 @@
-After our most recent conversation about this, I decided that I was going to do another pass of research focused on the ways that AI harms human cognition. If it does, to what degree, what is the science actually say about this? And what I found was that it depends. Go figure.
+Most of my argument has been that how someone engages with AI matters. I’ve written about this on the blog before: I think AI is a multiplier. My argument was that if someone was already lazy, impressionable, or easily influenced, AI could amplify those traits and create additional problems. On the other hand, if someone was already a deep thinker, self-critical in a productive way, they might already have some of the habits and self-checks that help prevent the problems my wife is worried about.
@@ -25 +25 @@
-One study showed that depending on the prompt, AI was an excellent study tool for things like math, but could also cause a massive decrease in the skill of the users where they would overestimate their capabilities and then perform much more poorly on tests of the material. Another study showed that AI assistance for colonoscopies had a similar effect. Granted, these are slightly different use cases than what I assume the average person uses AI for LOL.
+I should also define what I mean by AI, because that word seems to mean something different to everyone. Here, I’m talking about large language models. In my fairly informal description, they’re incredibly good guessing machines built out of the magic of statistics.
@@ -27 +27 @@
-But that got me thinking even more over the next couple days. I began to integrate all of this research that I had uncovered into the way that my harness functions.
+After our most recent conversation, I decided to do another pass through the research, specifically looking at how AI might harm human cognition. Does it? To what degree? What does the science actually say?
@@ -29 +29 @@
-And then last night, as I was getting the shower, I had a sudden epiphany. I've always taught people, especially at work, about effective ways to use AI, especially when I was in the chat era of my journey. And one of the things that I've described is the concept of context poisoning, which is the idea that because these are basically just models and they just guess whatever input you provide will inevitably affect the output.
+What I found was: it depends.
@@ -31 +31 @@
-So, sure, duh. But you're not the only one giving input to an AI model. There's also the system prompt. There's also the built-in skills. There's also whatever else is on the back end that we don't know about. Not to be conspiratorial or anything. And more importantly, there's the training data. There are so many ways that context can be affected, and therefore the output.
+Go figure.
@@ -33 +33 @@
-So it follows that you need to be careful about what you put into an AI system because it will affect the output in ways that may drive your decision making. It may point you towards solution A over solution B, or it may never even tell you about solution B, depending on how you asked. Additionally, if you weren't aware of solution B, then you've effectively cheated yourself unwittingly out of the knowledge of a thing.
+One study I read suggested that, depending on how the assistance was set up, AI could be an excellent tool for studying things like math. It could also leave students overestimating their capabilities and performing much more poorly when tested without that assistance. Another study, involving AI assistance for colonoscopies, raised a similar concern about skill loss.
@@ -35 +35 @@
-Now I realize that that's always a possibility, but one of the best things that I think AI does for us is allow us to more easily find the unknown unknowns. So oftentimes I will give it a very vague prompt or question, but with a prompt around it that instructs it to search widely into things that might not even necessarily be related so that I can get that information and judge for myself because, as we all know, these models are not to be trusted.
+[Add the math and colonoscopy studies and check the descriptions of their findings.]
@@ -37 +37 @@
-Now, that led to another thought going back to what my wife and her concerns and the research around them were since these statistical models are built off of human statements, human thoughts, human words. They are using or guessing into using human methods. And if we can poison their context window, then it follows that they could probably poison the human context window. You yet another bout of research. Turns out this is a thing that has been studied, and there is precedent for it I need to go look at the research for this part
+Granted, colonoscopies are a slightly different use case from what I assume the average person is doing with AI. lol.
@@ -39 +39 @@
-That is what prompted me to attempt to create an admittedly ham-fisted solution. My thinking is if the AI can no longer provide you with anything other than what you've specifically asked for, then maybe that will limit the amount of influence it has on the human decision-making context.
+Over the next couple of days, I kept thinking about what I’d read and began incorporating that research into how my harness functions. Then, last night, as I was getting into the shower, I had a thought.
@@ -41 +41 @@
-Now, I'll tell you, I'm writing, or rather, dictating, this article to a Claude Astra eye session in VS Code using handy and I can tell you right now from personal experience, keep in mind this is the first time that I've used null mode for anything real since I created it yesterday. That it absolutely forces you to actually think through everything you're doing.
+I’ve spent a lot of time teaching people, especially at work, how to use AI effectively. Even during the chat phase of my own use, one of the things I talked about was what I called “context poisoning”: whatever you put into the model affects what comes out.
@@ -43 +43 @@
-And yes, I know a few synthetic tests and some anecdotal evidence and a few carefully selected research papers or studies or retrospectives. Do not turn this into accepted theory.
+Sure. Duh.
@@ -45 +45 @@
-So that's why I published it publicly and bothered to write this article and post it on every social service that I have the stomach to use x.com can kiss my ass
+But you aren’t the only one supplying input. There’s the system prompt, any instructions or skills the system loads, and whatever else the product puts behind the scenes that we don’t necessarily see. Not to be conspiratorial about it. And beyond the immediate context, there’s the training data that shaped the model in the first place.
@@ -47 +47 @@
-I'm hoping that this gets out there widely enough to maybe give me some additional data points, but also hopefully to encourage more people to take the time to think and reason through things that they should care about.
+All of that affects the output. The way you ask a question can steer the response toward solution A rather than solution B. It might never mention solution B at all. If you didn’t know solution B existed, you’ve unwittingly missed the opportunity to learn about it.
@@ -49 +49,33 @@
-I'll link all of the studies and probably my research summaries near the end of this article. So you can go read them for yourself and give Null a try. I'm very interested in whether it makes any sort of difference at all
+I realize that’s always a possibility, but finding unknown unknowns is one of the things I find most useful about AI. I’ll often give it a vague question with instructions to search widely, including things that might not initially seem related. I want that information so I can judge it for myself, because these models aren’t something I’m willing to trust at face value.
+
+That brought me back to my wife’s concerns.
+
+These models are trained on human statements, thoughts, and words. Their responses can reproduce ways that humans communicate and influence one another. If we can “poison” a model’s context, could it do something similar to ours?
+
+Could it affect what I’m considering, what I’m overlooking, and how I’m arriving at a decision?
+
+That prompted another round of research. I found work that seemed relevant to that question, though I need to return to the sources before describing exactly what it establishes.
+
+[Add the research on AI influence and human decision-making here.]
+
+That’s what led me to attempt an admittedly ham-fisted solution: the null skill.
+
+My thinking was that if the AI stopped volunteering things I hadn’t asked for, maybe I could limit how much it influenced my own decision-making context. It would respond to the operation I actually requested, while leaving the things I hadn’t supplied unresolved.
+
+Maybe that would give me more room to think.
+
+I’m dictating this article into an AI session in VS Code using Handy, with null mode active. This is the first time I’ve used it for anything real since creating it yesterday. My experience so far is that it forces me to think through what I’m doing. I have to work through what I want to say.
+
+That’s a personal observation from one attempt. A few synthetic tests, some anecdotal evidence, and a selection of research papers do not turn my idea into an accepted theory. I don’t know whether this will make a meaningful difference beyond what I’ve noticed while using it to dictate this article.
+
+That uncertainty is part of why I published the skill and wanted to write about it. It’s also why I’m bothering to share this on every social service I have the stomach to use.
+
+X.com can kiss my ass.
+
+I’m hoping it reaches enough people to give me some additional data points. I also hope it encourages people to take the time to think and reason through things they care about.
+
+I’ll link the studies and probably my research summaries here so you can read them for yourself. Give null a try if you’re interested. I’m very interested in whether it makes any difference at all.
+
+[Add the null skill link, studies, and research summaries.]
+
+*How this one was made: I dictated the source material using Handy with null mode active, then asked AI to shape the dictation into an article. The research references are still placeholders in this draft.*
Edited draft → revised article
--- draft.md
+++ assisted-researched.md
@@ -1 +1 @@
-For roughly three months now, I’ve been heavily using AI agents. Before that, I used chats fairly regularly to create applications and small tools, copy-pasting back and forth between an IDE and Claude or ChatGPT.
+For roughly three months now, I’ve been heavily using AI agents. Before that, I used chats fairly regularly to create applications and small tools, copy-pasting back and forth between an IDE and Claude or ChatGPT. One of the things that came out of that progression is a skill called null. It asks the AI to reply “Received.” while I think out loud, then do the work when I specifically ask for something. I wanted to choose when it started contributing.
@@ -5 +5 @@
-I started with my understanding of software engineering and how software functions, then used agents to research and establish best practices. That became my first AI harness, which was essentially a set of Copilot instructions, some of which applied when a file matched a particular pattern. Those instructions have since migrated into skills, including an engineering assurance skill with different modes for reviewing work after design, before implementation, and after implementation.
+I started with my understanding of software engineering and how software functions, then used agents to research and establish best practices. That became my first AI harness, which was essentially a set of Copilot instructions, some of which applied when a file matched a particular pattern. Those instructions have since migrated into skills, including an engineering assurance skill for planning before substantial implementation and reviewing implemented work before release.
@@ -15 +15 @@
-She’s concerned about how much time I spend interfacing with AI. My wife isn’t particularly technical, but she also has very good reasons to be concerned about how much time I spend on anything. Say the words “Dota 2,” as in Valve’s online MOBA, then step back and watch her reaction.
+She’s concerned about how much time I spend interfacing with AI. She has very good reasons to be concerned about how much time I spend on anything. Say the words “Dota 2,” as in Valve’s online MOBA, then step back and watch her reaction.
@@ -21 +21 @@
-Our conversations forced me to consider more deeply what effect AI is having on me. Being the person I am, I felt I had some well-thought-out arguments for why the time and effort I’ve put into working with and understanding these systems wasn’t the same problem we’ve all seen discussed online and in news articles. I’m talking about things described as “AI psychosis,” along with deskilling and offloading cognition to the machine.
+Our conversations forced me to consider more deeply what effect AI is having on me. Being the person I am, I felt I had some well-thought-out arguments for why the time and effort I’ve put into working with and understanding these systems wasn’t the same problem we’ve all seen discussed online and in news articles. I’m talking about things described as “AI psychosis,” along with deskilling and offloading cognition to the machine. Those questions grew out of our conversation, but they don’t settle her concern about the time itself. Null doesn’t answer that either.
@@ -23 +23 @@
-Most of my argument has been that how someone engages with AI matters. I’ve written about this on the blog before: I think AI is a multiplier. My argument was that if someone was already lazy, impressionable, or easily influenced, AI could amplify those traits and create additional problems. On the other hand, if someone was already a deep thinker, self-critical in a productive way, they might already have some of the habits and self-checks that help prevent the problems my wife is worried about.
+Most of my argument has been that how someone engages with AI matters. I’ve written about [AI amplifying what was already there](https://probablyfine.dev/blog/written-by-no-one) before. I think AI is a multiplier. I was also making an argument about myself: I thought that being willing to question my own ideas and check what the model told me would help protect against the problems my wife was worried about. That was my reasoning, but it isn’t evidence that I’m protected. The studies discussed here don’t establish a protective personality profile or tell me what is happening to me. They also don’t establish a cause of psychosis.
@@ -25 +25 @@
-I should also define what I mean by AI, because that word seems to mean something different to everyone. Here, I’m talking about large language models. In my fairly informal description, they’re incredibly good guessing machines built out of the magic of statistics.
+I should also define what I mean by AI, because that word seems to mean something different to everyone. Here, I’m talking about large language models. In my fairly informal description, they’re incredibly good guessing machines built out of the magic of statistics. More precisely, models of this kind learn patterns during training and generate text by predicting tokens, which can be words or parts of words, conditioned on the text they’re given. That distinction between training and the current prompt matters. [Brown et al., 2020](https://arxiv.org/abs/2005.14165).
@@ -33 +33 @@
-One study I read suggested that, depending on how the assistance was set up, AI could be an excellent tool for studying things like math. It could also leave students overestimating their capabilities and performing much more poorly when tested without that assistance. Another study, involving AI assistance for colonoscopies, raised a similar concern about skill loss.
+I had read the papers. For this version of the article, I asked AI to check the studies I was referring to and fill in the references. We then went back and forth over the draft, including the difference between my original claims and the qualifications it had added.
@@ -35 +35,3 @@
-[Add the math and colonoscopy studies and check the descriptions of their findings.]
+The math study involved nearly 1,000 high-school students. Both GPT-4 tools helped them perform better during practice. But when the AI was taken away, the group using the less constrained tool scored 17% below the control group. The tool built around teachers’ tutoring guidance largely avoided that penalty, though it didn’t produce a statistically significant improvement on the unaided exam. Students using the less constrained tool also didn’t recognize how much worse their learning and performance had been. That’s a short-term learning result; it doesn’t tell me what months of using coding agents have done to me. [Bastani et al., 2025](https://doi.org/10.1073/pnas.2422633122).
+
+The colonoscopy study was a retrospective before-and-after comparison at four Polish centres. Across 1,443 procedures performed without AI, the proportion detecting at least one adenoma fell from 28.4% before AI was introduced to 22.4% afterward: six percentage points. That raises a concern about deskilling, but this observational comparison cannot establish that AI caused the decline. It also studied a medical detection tool, not a conversational language model. [Budzyń et al., 2025](https://pubmed.ncbi.nlm.nih.gov/40816301/?dopt=Abstract).
@@ -39 +41 @@
-Over the next couple of days, I kept thinking about what I’d read and began incorporating that research into how my harness functions. Then, last night, as I was getting into the shower, I had a thought.
+Over the next couple of days, I kept thinking about what I’d read and began incorporating that research into how my harness functions. At one point, as I was getting into the shower, I had a thought.
@@ -41 +43 @@
-I’ve spent a lot of time teaching people, especially at work, how to use AI effectively. Even during the chat phase of my own use, one of the things I talked about was what I called “context poisoning”: whatever you put into the model affects what comes out.
+I’ve spent a lot of time teaching people, especially at work, how to use AI effectively. Even during the chat phase of my own use, one of the things I talked about was what I called “context poisoning”: the information and framing you give a model can affect what comes out. I’m using that phrase informally here. A prompt influencing a response is normal model behavior; it doesn’t require malicious input or mean that every part of a prompt changes every answer.
@@ -47 +49 @@
-All of that affects the output. The way you ask a question can steer the response toward solution A rather than solution B. It might never mention solution B at all. If you didn’t know solution B existed, you’ve unwittingly missed the opportunity to learn about it.
+All of that affects the output. The way you ask a question can steer the response toward solution A rather than solution B. It might never mention solution B at all. If you didn’t know solution B existed, you’ve unwittingly missed the opportunity to learn about it. There’s a related, studied failure mode here: sycophancy. Sharma and colleagues found that assistants sometimes favored answers agreeing with a user’s stated beliefs over truthful ones. [Sharma et al., ICLR 2024](https://arxiv.org/abs/2310.13548).
@@ -57 +59 @@
-That prompted another round of research. I found work that seemed relevant to that question, though I need to return to the sources before describing exactly what it establishes.
+That prompted another round of research. The human “context window” is my analogy, not a claim that human memory works like an LLM. Two relevant papers address different parts of that question.
@@ -59 +61,3 @@
-[Add the research on AI influence and human decision-making here.]
+Glickman and Sharot studied 1,401 participants across experiments involving perceptual, emotional, and social judgments. Biased systems increased bias in people’s judgments, and participants underestimated some of that influence. Accurate systems could also improve independent judgments. These experiments used several kinds of systems, including hard-coded algorithms and image generation; they weren’t a test of extended conversations with coding agents. [Glickman and Sharot, published online in 2024, journal issue 2025](https://www.nature.com/articles/s41562-024-02077-2).
+
+For conversational language models, Salvi and colleagues ran a randomized study with 900 participants in short debates. GPT-4 given personal information was more persuasive than the human baseline. However, a September 2026 correction says its advantage over GPT-4 without personal information was not statistically significant. It also doesn’t tell us what happens over months of ordinary agent use. [Salvi et al., 2025](https://www.nature.com/articles/s41562-025-02194-6); [2026 correction](https://www.nature.com/articles/s41562-026-02588-0).
@@ -63 +67 @@
-My thinking was that if the AI stopped volunteering things I hadn’t asked for, maybe I could limit how much it influenced my own decision-making context. It would respond to the operation I actually requested, while leaving the things I hadn’t supplied unresolved.
+My thinking was that if the AI stopped volunteering things I hadn’t asked for, maybe I could limit how much it influenced my own decision-making context. The [skill’s instructions](https://github.com/maxfield-allison/null/blob/main/SKILL.md) ask it to respond to the operation I actually requested, while leaving the things I hadn’t supplied unresolved. While I’m thinking aloud, it should say “Received.” An explicit request permits the requested work, then it returns to quiet behavior.
@@ -65 +69 @@
-Maybe that would give me more room to think.
+Those are instructions, not a guarantee. Null doesn’t erase earlier context, remove the effects of training, override higher-priority instructions, or make the model neutral. An outline or critique I ask for can still change the direction I take. Staying quiet could also mean it withholds a suggestion I would have found useful. I still want that help when I ask for it. Whether choosing when it contributes reduces unwanted influence remains a hypothesis.
@@ -67 +71 @@
-I’m dictating this article into an AI session in VS Code using Handy, with null mode active. This is the first time I’ve used it for anything real since creating it yesterday. My experience so far is that it forces me to think through what I’m doing. I have to work through what I want to say.
+I dictated the original version of this article into a Codex session in VS Code using Handy, with null mode active, on September 23. It was my first real use of null. My impression was that it forced me to think through what I was doing. Later in the same conversation, I turned null off and asked for a review. I could decide when I wanted the model’s opinion, even though that doesn’t tell me how much it influenced me.
@@ -69 +73 @@
-That’s a personal observation from one attempt. A few synthetic tests, some anecdotal evidence, and a selection of research papers do not turn my idea into an accepted theory. I don’t know whether this will make a meaningful difference beyond what I’ve noticed while using it to dictate this article.
+That’s a personal observation from one attempt. The [repository’s testing note](https://github.com/maxfield-allison/null#testing) records 20 written behavioral checks, nine of which were run once each against an earlier version. Those check instruction-following, not cognitive outcomes. A few synthetic tests, some anecdotal evidence, and a selection of research papers do not turn my idea into an accepted theory. I don’t know whether this will make a meaningful difference beyond what I’ve noticed while using it to dictate this article.
@@ -71 +75 @@
-That uncertainty is part of why I published the skill and wanted to write about it. It’s also why I’m bothering to share this on every social service I have the stomach to use.
+That uncertainty is part of why I published the skill and wanted to write about it. It’s also why I want to share this on every social service I have the stomach to use.
@@ -75 +79 @@
-I’m hoping it reaches enough people to give me some additional data points. I also hope it encourages people to take the time to think and reason through things they care about.
+I’m hoping it reaches enough people to hear how it works for them. I also hope it encourages people to take the time to think and reason through things they care about.
@@ -77 +81 @@
-I’ll link the studies and probably my research summaries here so you can read them for yourself. Give null a try if you’re interested. I’m very interested in whether it makes any difference at all.
+The studies are linked above so you can read them for yourself. You can find [null and its instructions on GitHub](https://github.com/maxfield-allison/null). Give it a try if you’re interested. Tell me what you were trying to do and whether the waiting helped or just got in the way.
@@ -79,3 +83 @@
-[Add the null skill link, studies, and research summaries.]
-
-*How this one was made: I dictated the source material using Handy with null mode active, then asked AI to shape the dictation into an article. The research references are still placeholders in this draft.*
+**How this one was made.** I dictated the original using Handy with null mode active, then asked Codex to shape it into an article. I corrected two passages and clarified that I had read the papers. At my request, Codex checked sources, filled in research details, and wrote the added summaries and qualifications. After I turned null off, I asked for its critique and agreed to another editing pass. This revised article went through several exchanges; the original dictation and first edited draft are preserved for comparison. Codex also wrote the comparison document and built the page. The personal experience and reasons for creating null are mine; the added prose and implementation were written by Codex.

How this one was made. I dictated the original using Handy with null mode active, then asked Codex to shape it into an article. I corrected two passages and clarified that I had read the papers. At my request, Codex checked sources, filled in research details, and wrote the added summaries and qualifications. After I turned null off, I asked for its critique and agreed to another editing pass. This revised article went through several exchanges; the original dictation and first edited draft are preserved for comparison. Codex also wrote the comparison document and built the page. The personal experience and reasons for creating null are mine; the added prose and implementation were written by Codex.

Compare this passage

AI-assisted change note

Read the full comparison document

related posts

comments

Your saved places

Keep a place in an article or case study, then come back to it. These bookmarks stay in this browser, on this site.

In this page