Speed and deception challenge the viability of human oversight.
Biden’s plan says the President will decide when to use military AI, ensuring accountability through human involvement. But this isn’t always realistic. Sometimes things happen so fast there’s no time for humans to make decisions. Luxury cars already use AI to tighten seat belts when sensing a crash. Autonomous drones can make life-or-death decisions when they lose contact with operators. Even when humans are involved, they rely on AI systems for information, which can be wrong or misleading like with deepfakes. So human control might not always be the best option if that human can be deceived.
— Biden’s AI Plan And Why Central Control Might Not Work · Forbes