AI Vulnerability: Who Pays for the Damage Caused by Artificial Intelligence?

When AI Silently Breaks Things, Who Pays?

The integration of artificial intelligence into business operations is bringing new challenges to the insurance sector. David Halbreich, a partner specializing in insurance recovery at Reed Smith, has analyzed how AI companies should handle coverage gaps that emerge as the sector grows.

The Challenge of "Straddle Claims" Post-Merger

One of the most common problems Halbreich encounters involves so-called "straddle claims," or damage claims that fall between the retroactive ("tail") and prospective ("go-forward") coverage after a merger. This issue is particularly relevant in the AI sector, where merger activity is intense. Directors and officers liability insurance (D&O) and errors and omissions (E&O) policies are essential for AI companies. These policies are typically "claims-made," meaning they cover damage claims presented during the coverage period, unlike "occurrence"-based policies that cover damages arising from an event that occurred during the coverage period, regardless of when the claim is presented.

The Coverage Gaps

When a target company is acquired, it is customary to obtain an extended reporting period coverage (also known as "tail" or "runoff" coverage) that extends the window during which damage claims can be reported under the policy. The acquiring company will generally procure a new policy (often called "go-forward" coverage) that takes effect on the closing date or around it and is designed to respond to damage claims arising from behaviors that occurred after the transaction. In theory, this framework seems to provide continuous protection for historical liabilities (under tail coverage) and future behaviors (under go-forward coverage). However, problems can arise when a damage claim does not clearly fall into one of the two periods. Tail policies often contain broad exclusion clauses for damage claims involving any behavior subsequent to the cutoff date, but such claims might not be covered by the go-forward policy either, to the extent they also involve pre-merger behaviors.

The Issue of Governance Representations

Underwriters are now asking for governance documents such as bias test logs, human-in-the-loop protocols, model cards, and assessment results. When these responses are incorporated into the policy by reference, at what point does a governance representation stop being an underwriting input and become a prior condition or a warranty that the carrier can use to deny a damage claim? Halbreich suggests that stakeholders would benefit from adding more specificity. Adding requirements for knowledge/intention and materiality of false statements can prevent the incorporation of the question into the policy from becoming a strict warranty. Stakeholders should also seek limitations on the consequences of false statements, with coverage limited only to those covered individuals responsible for the false statements, or limited only to the company if certain identified individuals knew of the false statement. Stakeholders should also ensure that the policy specifies that false statements in the application will not be grounds for rescission of the entire policy.

Managing AI Usage Questions

Questions about AI usage in insurance applications are often answered by a risk manager who does not know what each product team is executing. How should a company build the internal process to answer these questions and who should sign? Halbreich emphasizes that this is a problem organizations have had to address before it arose in the context of AI. The initial aspect of addressing this issue is simply establishing the necessary protocols. For AI usage, there may be policies related to which specific agents or products can be used, in what contexts they can be used, what information can be provided to them, etc. However, the real challenge is ensuring that everyone in the organization is compliant and that they are. Halbreich advises risk managers to establish an organized process several months before the renewal date of the existing program that informs the organization of the upcoming renewal, emphasizes the importance of the renewal for understanding the different products the company is executing, and informs that risk management will follow up with individual teams to obtain the most complete picture possible.

Model Degradation and Claims-Made Policies

Model degradation is gradual. A claims-made policy requires an act, an error, or an omission and a damage claim. When a client discovers eighteen months of output that silently deteriorates, when does the clock start and how do the damage claim provisions help or hurt the insured in this scenario? Halbreich explains that under a claims-made policy, a coverage event occurs when an act, error, or omission occurs and a damage claim is presented during the coverage period. However, model degradation is a gradual process and it can be difficult to determine exactly when the act, error, or omission that initiated the degradation occurred. This can create challenges for insureds in determining when to present a damage claim and when coverage is available. Halbreich emphasizes the importance for companies to fully understand their insurance policies and the representations they are certifying in insurance applications. One should not sign something just because it is assumed to be "standard". If what an insurer asks is too broad, or not something a person can reasonably attest to, the company should work to modify the representation into something more reasonable.