AI can mistakenly see cancer in medical scans after tiny image tweaks

https://www.newscientist.com/article/2197369-ai-can-mistakenly-see-cancer-in-medical-scans-after-tiny-image-tweaks/

A medical scanner

Some medical AIs are easily tripped up

Science Photo Library/Getty

Medical artificial intelligence breaks a little too easily. Although AI promises to improve healthcare by quickly analysing medical scans, there is increasing evidence that it trips up on seemingly innocuous changes.

Sam Finlayson at Harvard Medical School and his colleagues fooled three AIs designed for scanning medical images into misclassifying them by simply altering a few pixels.

In one example, the team ever so slightly altered a picture of a mole that was first classified as benign with 99 per cent confidence. The AI then classified the altered image as malignant with 100 per cent confidence, despite the two images being indistinguishable to the human eye.

Advertisement

The other two AIs were for detecting damage to the eyes from diabetes in retina scans and spotting a collapsed lung from chest X-rays.

A strange turn

Rotating images can similarly confuse AI and could occur naturally in day-to-day practice. Even if the chance of this happening is low, if medical AIs become widespread, it could lead to many cases of misclassification.

There may also be incentives for people to doctor images this way. If dermatologists were to get reimbursed only for removing a mole by insurance companies if an AI agreed that it was malignant, there could be an incentive to alter borderline cases to ensure payment for more procedures, says Finlayson.

It isn’t always clear what factors AI uses for decision-making, which can cause problems when the tools are used in the real world.

Another recent study found that an AI designed to detect hip fractures was using the imaging device model and the patient’s age to make its prediction, rather than focusing on the fracture itself.

The model was a good proxy for how sick someone was, because it could reveal if the scanner used was a portable version and therefore if the person was too sick to visit the hospital.

Current efforts to design AI that are better at defending against attacks tend to come at the cost of being less accurate, says Finlayson.

Journal reference: Science, DOI: 10.1126/science.aaw4399

More on these topics: