The insurance company secretly spied on the client using a drone, and then canceled his policy

Albert Fox Kahn is the founder and executive director of the Surveillance Technology Oversight Project (STOP), a New York-based civil rights and privacy advocacy group. He encountered a glaring mistake that nearly cost him his insurance. Insider recorded Fox Kahn's story in first person.

My insurance broker left me a frantic voicemail saying that my home insurance had lapsed. I felt uneasy, I felt naked. Suddenly, any leak, fire, or tree branch falling on a century-old house in the Hudson Valley could wipe out my bank account. The house has been in my family for almost 40 years. How did I let this happen? Did I forget to update my credit card? Missed a payment? Did you do something wrong with your insurance? But when I checked my Travelers entries, I couldn't find anything.

A few hours later, my panic gave way to bewilderment. When I finally contacted my insurance broker, he gave me the reason why Travelers had canceled my insurance. The company used AI-powered drone surveillance.

Insurance company drones found... moss

I take privacy and surveillance extremely seriously. So seriously that he founded one of the leading think tanks on this topic, the Surveillance Technology Oversight Project. I study surveillance threats across the country for a living, but I had no idea that my insurance company was using my insurance dollars to spy on me. Travelers not only uses aerial photography and AI to monitor the roofs of its clients, but also patented this technology. In fact, almost 50 patents. This may not be the only insurance company spying on you from the sky. It didn't just seem creepy, it felt wrong. Literally wrong: there was nothing wrong with my roof.

On the subject: Housing, cellular communications, insurance, bank account for new immigrants in New York: the personal experience of a Russian

I'm a lazy homeowner. I hate gardening and don't clean as often as I should. But I still take care of the essentials. Whether it's upgrading your electrical system or installing new HVAC systems. I try to make sure my home is safe. But for Travelers' AI, my laziness was too much of a risk to insure. Its algorithm detected no problems with the foundation or leaking pipes. As my broker said, the ominous threat that canceled my insurance was nothing more than moss.

Where there is moisture, there is moss. If you leave a huge amount of it for a long time the roof, this can compromise the lifespan of the roof. A small amount is generally harmless. However, dealing with it couldn't be easier. Of course, I could have removed the moss earlier, but I didn't. It continued to grow between the cracks. Finally, in June, a few weeks before it became known that my roof was under surveillance, I went to the hardware store. There I spent $80 on a moss killer, at home I connected a white bottle of chemicals to a garden hose and sprayed it on the roof. The whole job took about five minutes. After a few days, much to my relief, the moss began to die. I thought that was the end of a completely forgettable story.

AI and insurance

Who knows, maybe if I had done this a month earlier, Travelers technology would never have flagged me. She would never say that I was an insurance risk. But one of the deep frustrations of the age of artificial intelligence is that as companies and governments track more and more details of our lives, we rarely know we're being watched. At least until it's too late. There is no way to know exactly how many other Travelers customers have been targeted by the company's surveillance program. I'm definitely not the first.

In February, an ABC affiliate in Boston notified a customer that she would not be renewed if she did not replace her roof. The roof was within its expected service life. The client did not have any problems with leaks. However, she was told that without a roof replacement she would not be insured. A client said she was billed $30 to replace the slate on her roof. The roof, according to experts, could last another 000 years.

Insurers have every incentive to be overly cautious in how they build their AI models.

No one can use artificial intelligence to know the future. You train technology to make guesses based on roof color changes and grainy aerial photos. But even the best AI models will make a lot of incorrect predictions. Especially when you're trying to make assumptions about the future of radically different roof designs on countless buildings under varying conditions. Insurance companies will have huge incentives to choose against the homeowner every time.

Think about it: every time AI greenlights a roof that actually has something wrong, the insurance company foots the bill. Each time this happens, the company can add that data point to its model and train it to be even more risk averse. But when homeowners are threatened with insurance cancellations, they foot the bill for repairs even if no repairs are needed. If a homeowner in Boston throws away a slate roof that could have lasted another 70 years, the insurance company will never know that it was mispriced. She never updates the model to be less aggressive on similar homes.

Unanswered Questions

Over time, insurance companies will have every incentive to make their models more and more ruthless. This threatens more Americans with loss of coverage and potentially leads to millions or billions of dollars in unnecessary home repairs. Insurers face mounting losses due to the climate crisis and inflation. The pressure to push unnecessary preventative maintenance on customers will only increase.

The confusing conclusion to this whole story is what Travelers said when I approached them with a detailed list of fact-checking questions. In response, the representative sent a brief rebuttal:

“AI analysis/modeling and drone surveillance are not part of our underwriting decision-making process. When available, our underwriters can reference high-resolution aerial photographs as part of a holistic view of the property’s condition.”

What exactly is considered the “underwriting decision process”? And if AI and drones don't actually impact customers, then why file so many patent applications such as "Systems and Methods for Analyzing Roof Wear Using Artificial Intelligence (AI)"? It seemed that the company was trying to sit on two chairs. She brags about using the latest and greatest technology and avoids responsibility for mistakes. When I asked Travelers these additional questions, the company did not respond.

Luckily, my roof isn't going anywhere, at least not yet. A few hours after my story with Travelers began, the situation was resolved - the company admitted that it had screwed up, but did not admit that its AI was wrong in flagging me. But she revealed the reason I couldn't find my cancellation notice: the company never sent it.

Travelers may have invested huge sums in neural networks and drones. But apparently it never updated its billing software to handle the basics reliably. Without notice of non-renewal, she could not legally cancel coverage. Bad advanced technology failed me; bad underlying software saved me.

Part of what's unsettling about this whole episode is how opaque it was. When Travelers flew a drone over my house, I never knew. When she decided I was at risk, I had no way of knowing. As more and more companies use increasingly opaque forms of AI to determine the course of our lives, we are all at risk. AI could give companies a quick way to save some money. But when these systems use our data to make decisions about our lives, we ourselves are at risk. As annoying as it can be to deal with a human insurance agent, it is clear that AI and surveillance are not the right substitute. And unless lawmakers take action, the situation will only get worse.

The reason I still have insurance is because of simple consumer protection laws. New York State does not allow Travelers to cancel my insurance without notice. But why do we even allow companies like Travelers to use AI against us without any protection? A century ago, legislators saw the need to regulate the insurance market and increase policy transparency.

We now need updated laws to protect us from AI attempts to decide our destinies. If this does not happen, the future looks worrying. Insurance is one of the few guarantees that protect us from the risks of modern life. Without protection from AI errors, algorithms will take away what little peace of mind our policies give us.