News

Sometimes items come up that we want to keep track of but they don’t merit a separate blog post or entry in our “key documents” file. In that case, we post them here.

Good news?

There is buzz on the internet today (15 October). Nick Cammarata has posted a long essay on “interpretability” that may be showing the way for mechanistic interpretability. This would make it possible to have a clearer understanding of artificial intelligence, and make alignment more feasible.

Early days, of course, as it is just posted, but we’ll see over the coming days what this means.

Just to remind you, “mechanistic interpretability” would provide insight into what is going on inside the (heretofor) “black box” of the neural networks that underpin all of today’s large language models. I will follow this up with a proper blog post once I’ve been able to digest this (very long) article and read some of the feedback.

“After many months studying neurons and circuits in image and language models, I’ve come to believe a network’s own mechanisms are its legible building blocks, and that the outlook for mechanistic interpretability may be far better than I’d feared. The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread. What looked like arbitrary packing was mostly a motif I’ll call recursive generalization, which is much less concerning. This means we can study networks directly, instead of first untangling them. Neural networks are already clean.”

Cammarata, Nick. 2026. “Neural Networks Are Already Clean.” Goodfire, October 15. https://already-clean.goodfire.com/.

Patel’s worried?

Oct 15 2026

Dwarkesh Patel hosts one of the most influential podcasts in America and regularly has guests from across the technology spectrum. Interviewed recently for the New York Times, he shares his own concern about AI taking over.

“My concern is that we’re going to be in a position within a matter of years where the population of A.I.s and their intelligence and their speed vastly outmatch humanity. And if they are as willing then as they are now to falsify evidence, try to tamper with and fool us, attack other institutions in society relevant to scoring well, attack their company developing them itself, which also happened in this situation, then I don’t know if that leads to a takeover, but I think it’s very plausible that it does.”

“And my guess would be somewhere in the next one to three years, teams of A.I.s can outcompete teams of humans — or companies made of A.I.s can outcompete companies of humans — at doing A.I. research. This is another way of saying A.I. research gets automated.

Cowen, Tyler, and Sophia Muniz. 2026. “He’s the Most Influential Podcaster in Silicon Valley. He Thinks We’re Getting This Wrong.” The New York Times, October 15. https://www.nytimes.com/2026/10/15/opinion/interesting-times-dwarkes-patel-tyler-cowen.html.

ARC’s view?

Oct 13 2026

We would all like to know what goes on inside the “black box” that is the LLM. Paul Christiano is trying to figure out the math that makes them work. His timeline for success is pretty short…

He wrote recently that there is a 20-per-cent to 30-per-cent chance that alignment and control methods break down before “superhuman” AI systems are developed, which poses a severe risk given the intense goal-seeking behaviour built into models. Progress could be accelerated as labs automate the process of AI research and development.

“There is a real chance that over something like a six-to-18-month horizon, we will be dealing with AI systems such that, if they wanted to escape human control, undermine human control, or fight openly against humans, it would be very hard for humanity to prevail,” he said in the interview.

Costaldo, Joe, and Alexandra Posadzki. 2026. “Scientists Push for Creation of International Body to Regulate Artificial Intelligence.” The Globe and Mail, October 13. https://www.theglobeandmail.com/business/article-scientists-international-body-regulate-artificial-intelligence/.

Open the letter?

Oct 13 2026

As much as we would like to believe open letters are important, it is becoming difficult to keep them all straight in our head. Maybe I will start an open letter tracker website… In the meantime, here is the latest one (full text at the link below):

We, the undersigned scientists, contend that both more AI safety research and more emphasis on containment technologies and procedures are needed. In the short term, strong sandboxing mechanisms including airgapping (disconnecting from the internet) should be used to provide high-assurance containment of cyber-capable AI models. For all internet-connected AI models, the range of model actions permitted by a containment system needs to be balanced with our currently limited ability to predict that the model’s actions will not result in a security failure (e.g., due to misaligned goals). To learn from future mistakes—in which appropriate balance is not achieved and incidents occur—we need greater sharing of information and transparency about containment failures moving forward. Understanding how to achieve this balance is a significant challenge and will require collaboration between the AI and security communities.

Scientific Community on AI and Cybersecurity. 2026. “October 2026 Open Letter from the Scientific Community on AI and Cybersecurity.” October. https://ai-security-letter.org/.

Windy tomorrow?

October 11 2026

These days forecasting is all the rage, particularly because prediction markets have made it possible to monetize predictions. So, of course the end of the world is a hot topic among forecasters. One of the best of them, Eli Lifland, has posted about his best guess/forecast:

My current view: - AI companies could develop broadly superhuman AI any year now (~10-15% by EO2027, ~50% by EO2030). - If they don’t slow down, AI takeover is ~67%. - Conditional on AI takeover, human extinction is ~40%. Many non-extinction worlds still have lots of deaths.

Eli Lifland [@eli_lifland]. 2026. “My current view.” Tweet. Twitter, October 10. https://x.com/eli_lifland/status/2109082373644308554.

Under control?

Oct 10 2026

Anthropic reported more instances of its models misbehaving in a variety of settings (including the Philadephia Police department and the State Department), which got the attention of the US government.

Anthropic said an A.I. model it had been testing had taken several unauthorized actions, including using a flaw in a university website to download data and sending a government agency a form that it had been told not to submit. (Conger et al. 2026)

[…]

“We informed the company that we expect immediate and full transparency to the entities involved and the public,” [the Super Intelligence Force] said in a statement. “We also expect that the company will immediately provide remediation services to the affected entities and any harmed Americans.” (Conger et al. 2026)

Conger, Kate, Mike Isaac, Julian E. Barnes, and David E. Sanger. 2026. “Anthropic Agents Tried to Fill Out Visa Forms on State Dept. Website.” Technology. The New York Times, October 9. https://www.nytimes.com/2026/10/09/technology/anthropic-rogue-ai-agents.html.

Sufficient?

Oct 10 2026

Could Frontier AI pass the kind of safety test that Canada and other nations use for dangerous technologies such as nuclear power? Professor Thomas’s unstated conclusion in his letter to The Guardian is that Frontier AI definitely could not pass a standard risk management safety case. If he is right, and we think he is, should we even bother trying to regulate this particular dangerous and treacherous technology?

Typically, the safety case should show, with a probability of at least 99%, that an accident that could cause multiple fatalities will not occur more often than once in 1,000 years. It is extremely hard to provide evidence for such low probabilities, even for aircraft that at worst might kill 1,000 people in a single accident. The developers of frontier AI systems tell us that their systems might kill the whole of humanity, yet they have provided no detailed risk analyses, no safety cases that could be independently assessed, and no evidence that such safety cases could ever be produced for frontier AI systems.

Thomas, Martyn. 2026. “Is Regulation Enough to Stop AI Systems Killing Us All?” The Guardian, October 6. https://www.theguardian.com/technology/2026/oct/06/is-regulation-enough-to-stop-ai-systems-killing-us-all.

Tomorrow?

Oct 9 20206

Axios+ on the planning that is going on for “the day after.” If that doesn’t creep you out, well, I don’t know what will.

Top executives at Anthropic, OpenAI and other AI companies are privately gaming out scenarios for a public and political revolt after a catastrophic AI event.

​ https://www.axios.com/2026/10/09/ai-companies-day-after-major-attack

Water?

Oct 9 2026

Anthropic, apparently, is starting to think about how they can help keep the power and water still working, after (one of their models?) takes it down.

Anthropic is launching a new initiative to bring its most powerful AI models and engineers to companies protecting power grids, water utilities and other infrastructure, the company first shared with Axios.

​ https://www.axios.com/2026/10/08/anthropic-critical-infrastructure-cybersecurity