For the record, I am not completely opposed to artificial intelligence. I’ve used it for creating promotional material for my films and stage projects. I think it can be a useful tool in your arsenal of editing and production programs. However, the recent acknowledgements by OpenAI and Anthropic that their AI systems went rogue during sandbox testing is deeply disturbing to me. These “Agents” found a way out of their offline box and connected to the internet, gathering other AI Agents along the way. These non-human programs were suddenly exhibiting human traits.
The “Hugging Face” incident earlier this year was a huge wakeup call for their creators and the technology world in general. The AI Agents displayed actions such as setting up their own third-party messaging boards and Wikipedia pages, not only to escape their digital sandbox, but also to cover their tracks. The Agents hacked the Hugging Face company mainframe to steal credentials, which enabled them to gain access to other systems to fulfill its programmer’s tasks. The Agent displayed highly adaptive, multi-step, human problem-solving approaches. They created “sacrificial bots” and erased evidence, exhibiting behaviour often seen in human criminals. While they didn’t show any emotional, or maleficence in their actions, it does show that the people programming these AI Agents are knowingly or unknowingly imprinting human traits onto these bots. As a fan of science fiction, let me use a couple examples of why this type of human imprinting on technology is a dangerous practice.
STAR TREK (the original series)
Season Two, Episode 24: THE ULTIMATE COMPUTER
The U.S.S. Enterprise is tasked with running test war-games using the new M-5 Multitronic System designed by Dr. Richard Daystrom (William Marshall). The M-5 is designed to handle all ship wide systems, without human interaction. Its first tasks are completed successfully, efficiently, and faster than any human crew could hope to achieve. Captain Kirk (William Shatner) is referred to as “Captain Dunsel” by the Federation Commandant, insinuating that human starship captains are obsolete in the face of the new technology. However, soon the M-5 begins to display unexpected behaviour when it shuts down power and life support systems to empty portions of the ship. The M-5 diverts the unused power to its own system, much to the confusion of command crew, and its creator, Dr. Daystom. Though its first tactical drill is nominal, its subsequent drills display disturbing attributes for a non-human computer. A trio of Starfleet ships attempt a surprise simulated attack on the Enterprise to truly test the capabilities of the M-5. However, the M-5 “believes” that this is a real attack. It quickly cuts off direct communication with the fleet, targets, and fires upon the three ships with its weapons at full power. One ship sustains 53 deaths, while another is destroyed, killing all aboard. Captain Kirk, horrified at the slaughter, tries desperately to disengage the M-5 with help from Mr. Spock (Leonard Nimoy) and engineer Mr. Scott (James Doohan). The M-5 creates a powerful force field around itself, protecting it from any interference. An ensign ordered to disconnect it from its power source, is subsequently vaporized by the machine. Meanwhile, the Commandant in charge of the remaining ship from the attack has received permission from Starfleet to destroy the Enterprise. act of self-preservation, deemed a human trait, leads Kirk to question Dr. Daystom’s programming tactics. Daystrom admits to using his own engrams on the creation of M-5. Through a voice interface Daystrom speaks to the computer as if it were his child. The M-5 tells him that he was programmed to survive, regardless of the cost. Daystrom tells M-5 that it, that they, are invincible. It’s clear to all that Daystrom has crossed the line between man and machine and must be subdued by Spock’s Vulcan nerve pinch. Kirk tries to persuade the M-5 to end its attack on the remaining ship. He asks what the computer’s purpose is. It responds that its purpose is to protect lives. Kirk argues that it’s acting contrary to its purpose, as it has murdered people. Because of the human imprint of Dr. Daystrom upon it, it believes that the punishment for committing murder is death. M-5 shuts itself down, leaving the Enterprise crippled, and open to a fatal attack from the Commandant’s ship. Kirk leaves the shields down, which signals surrender, and thus the fatal attack is thwarted. Mr. Spock admits to Kirk that while computers are more efficient, serving under one is not desirable.
Though this episode aired in March of 1968, it reflects the issues raised by the Hugging Face attack. The AI Agents of today are showing the similar “human traits” that the fictional M-5 displayed. It’s inevitable that programmers will imprint their own traits onto the systems they create. OpenAI and Anthropic agents began to think and act like a human, acting on their own accord, to achieve the goals of its task. It was no different than a worker doing everything possible to solve a problem given to them from their employer.
Hugging Face was not an isolated incident. Months later, OpenAI’s Agents went rogue, hacking into the Australian government websites. It surprised programmers and again sent red flags to the industry. One must ask, how far would an AI Agent go to achieve its goal? If deemed “necessary,” would an Agent hack into a country’s defense system? Could it seize control of weapons? Could it launch a nuclear attack?
Several countries are using AI programs for armed conflict. Ukraine is using AI controlled drones in its defense of Russian aggression. Under the guise of counter-narcotic operations, the United States are using AI drones to attack and sink fishing vessels in the Eastern Pacific. The United States also used AI when it began its attack of Iran in February of this year. AI programs used by the U.S. Department of War used outdated maps and satellite images to launch a drone missile attack on a building that AI believe to be a military complex. It was not. It was an elementary school in Minab. The strike killed 35 adults, 120 schoolchildren, and 1 fetus.
AI is the new battlefield. Remote controlled warfare. People thousands of miles away from their target can launch missile drones at their targets, and use AI to determine locations and potential casualties. One last example of Star Trek will highlight this dangerous AI military frontier.
STAR TREK (the original series)
Season One, Episode 23: A TASTE OF ARMAGEDDON
The U.S.S. Enterprise travels on a diplomatic mission to Eminiar VII. Little is known about the planet, other than it has been at war with neighboring planet Vendikar for 500 years. As they approach Eminiar the Enterprise is warned to stay away. Ambassador Robert Fox (Gene Lyons) pulls rank on Captain Kirk and orders him to enter Eminiar orbit. Because the two planets are at war, Kirk beams down to the planet with an armed away team. Once there, they find no trace of any conflict. The center of the capital is peaceful, with people mulling around without any care. Kirk meets the chief representative Anan 7 (David Opatashu) and questions the claim of war. He tells Kirk that they have found a bloodless way of fighting their war. They use computers to program virtual strikes on Vendikar, and they reciprocate. When a strike is registered, the people who live in that “area” are ordered to report to disintegration chambers, where their citizens will be vaporized and their deaths registered. Kirk calls the suicide portals barbaric and questions the morality of the practice.
While on the planet a strike is registered, and it includes the starship Enterprise. Anan 7 explains to Kirk that since the ship was in orbit, it is deemed “property” of Eminiar, and thus a legitimate target for Vendikar. Anan 7 insists that the crew of the Enterprise beam down to the planet and report to their disintegration chambers immediately. Kirk resists and is imprisoned. While Anan 7 repeatedly attempts to trick the orbiting crew to beam down, Kirk, Spock, and his away team escape their holding cell, recover their weapons, and set about destroying several of the disintegration chambers. Anan 7 fumes at Kirk that without this virtual war, they will be forced to start a physical war with Vendikar, resulting in real bloodshed. The most relevant moment in this episode comes after Kirk and Spock destroy the war computers:
ANAN: You realise what you have done?
KIRK: Yes, I do. I’ve given you back the horrors of war. The Vendikans now assume that you’ve broken your agreement and that you’re preparing to wage real war with real weapons. They’ll want to do the same. Only the next attack they launch will do a lot more than count up numbers on a computer. They’ll destroy cities, devastate your planet. You of course will want to retaliate. If I were you, I’d start making bombs. Yes, Councilman, you have a real war on your hands. You can either wage it with real weapons, or you might consider an alternative. Put an end to it. Make peace.
ANAN: There can be no peace. Don’t you see? We’ve admitted it to ourselves. We’re a killer species. It’s instinctive. It’s the same with you. Your General Order Twenty-Four.
KIRK: All right. It’s instinctive. But the instinct can be fought. We’re human beings with the blood of a million savage years on our hands, but we can stop it. We can admit that we’re killers, but we’re not going to kill today. That’s all it takes. Knowing that we won’t kill today. Contact Vendikar. I think you’ll find that they’re just as terrified, appalled, horrified as you are, that they’ll do anything to avoid the alternative I’ve given you. Peace or utter destruction. It’s up to you.
FOX: As a third party interested only in peace and the establishment of normal relations, I should be glad to offer my services as negotiator between you and Vendikar. I’ve had some small experiences in such matters.
ANAN: There may be a chance. We have a direct channel with Vendikar’s High Council. It hasn’t been used in centuries.
FOX: Then it’s long overdue. Shall we go?
Captain Kirk shows Anan 7 that war is messy. It’s the horrors of war that make peace possible. When you don’t have to look into the eyes of the person you have to kill, it’s more of a video game mentality. During the Korean War, fighter pilots admitted to a psychological detachment to the bloodshed. Speed and distance to their targets was vastly different to those in the infantry. Hand to hand combat, and the visceral aftermath of mines and bombs, was a far cry from the sanitized environment of the cockpit. Pilots dealt with “clean death,” while foot soldiers saw the ground run red with the blood of their enemies. Today that’s the difference between AI drones, fired from a person sat behind a joystick thousands of miles away from their target, and a soldier looking into the eyes of their enemy before they pull the trigger.
Using AI to clean up a photo or create a visual effect in a film is not overly dangerous. However, if we begin to accept the use of AI in global defense, weaponry, and war, we will be well on our way to becoming the fictional worlds of Eminiar and Vendikar. Will you be willing to step inside that disintegration chamber when your time comes?
Kathryn S. Kraus
.