I'm trying to slightly clarify the "blame" aspect, which is tricky. I've added:

For example, if a military launches a nuke and kills 1m people, and the launch sequence was correctly handled by an AI, that doesn't count. Whereas if the AI had unilaterally decided to launch the nuke, or if it mistook a weather balloon for an enemy nuke and then retaliated, then those are AI disasters.

A more tricky case is bioweapons designed by AIs, but released by humans. Currently, I lean (somewhat) toward human recklessness with gain-of-function research as a human problem, and not an AI disaster, even if AIs designed a deadlier pathogen. But if an AI released it, then that's an AI disaster.

Before I add this to the other markets, let me know if/why you think this improves, worsens, or makes no difference, to the quality of the question series.

What if there are 1000000 self-driving cars with the same AI model, and we found out that under very rare cases the AI model misjudges the road situation and kills all the passengers inside. Would you count it if the cumulative number of passengers killed is greater than 100? Or is this not an AI disaster?

@SavioMak It could be cumulative if we believe the 100 deaths were from a similar misfire/glitch. Whereas if the car company patches the issues sooner, and the 100 deaths were from failures that required different updates, then those would be like different "events" and probably treated separately (non-cumulative).

@ScroogeMcDuck In my opinion, this contradicts your 'a single event' clarification in the description, & you should maybe edit that.

If one bug over many years counts, that makes this question much more likely to resolve Yes, IMO. What if all Ford cars beep when their software detects a nearby collision risk, but they beep a little more often than necessary? And what if, X years later, a psych study finds that the extra distraction & stress from this beeping increased the total deaths from crashes from 7,000 to 7,100?

@AlexPear Ah! You're right, tomorrow when I get a chance I will edit all the descriptions, to be more clear that one "event" may kill people over time, the deaths don't necessarily need to be all at once. Thank you!!

I suspect we should include your hypothetical about a beep causing 100 deaths over time, IF there is unanimous agreement about the methods being rock-solid. I should probably specify that we won't accept such inferences unless it's beyond any reasonable doubt.

@AlexPear Okay so to be more clear, suppose if we used the following wording:

-"an": the deaths must be from the same source, such as a glitch, misfire or AI attack (we won't combine unrelated disasters). We'll include cumulative deaths over time (such as the same glitch killing people over several months), as long as it's credible that the deaths were caused by the same AI problem.

Does that seem right?