This article was written by imbricator and edited by Congress and Shymander of the MAL Articles Club.
Interested in writing or editing for us? Click here!
"Am I still to create the perfect system?" (Screencap from the film TRON: Legacy)
We're all aware of the stories about machine sentience that come as warnings. 2001: A Space Odyssey's HAL prioritizing his directive to maintain the psychological stability of the humans he works with to the point that he'd kill them before communicating disruptive information; WALL-E's AUTO literally following his directive to prevent humans from returning to Earth and using force on them as a result; TRON: Legacy's CLU developing an interpretation of a perfect world within TRON, one that diverged from his creator's.
In these famous speculative sci-fi tales, one thing regarding the antagonists is constant: they all believe that they are doing the right thing.
No matter how unprecedented the development of a computer program like CLU or HAL might be, these programs all started out as 0s and 1s and, like any other piece of code, were first defined by inviolable directives embedded in the fundamental workings of their existence. In the above cases, we see programs prioritizing their directives and basing their actions around fulfilling them.
Chamber, the AI continually supporting the protagonist of Suisei no Gargantia, also has a directive like HAL, AUTO, and CLU that's specific to his situation (in other words, specific to the plot of the story).
"I am a Pilot Support Enlightenment Interface System." This is Chamber's most-repeated line, a statement he constantly communicates to his pilot and Gargantia's main character, Galactic Alliance Lieutenant Ledo.
“By helping you achieve results, I fulfill my purpose of existence."
This guy is such a bro. Best wingman ever.
The premise of Suisei no Gargantia is that humanity has been locked in an everlasting war with giant squid-like aliens called Hideauze in space. Humanity fights the Hideauze with soldiers piloting mecha robots, each of which has the capacity for independent, critical thought. These mecha all identify as Pilot Support Enlightenment Interface Systems. Their directive, specific to the Suisei no Gargantia humans-against-aliens storyline, is to ensure the success of their pilots.
Ledo is battling the Hideauze when he's given a retreat order, but his sync with the transportation wormhole goes wrong and he ends up on a watery planet. Ledo is shocked to discover that this planet is the Earth of legend, that life can be sustained here, and that human life here is frustratingly inefficient compared to life in the Galactic Alliance. As a soldier in an unfamiliar place, all Ledo knows to do is wait for orders. While waiting for his distress broadcast to be heard, he assimilates into Earth society to the best of his ability, helped along every step of the way by his robot's Pilot Support Enlightenment Interface System, Chamber. Chamber, with his ability to analyze massive amounts of data to produce near-instantaneous conclusions, is pretty indispensable, especially considering that Ledo can't speak the same language as the Earth residents for much of the series.
Always nice to see the reactions of futuristic societies to the custom of meat-eating.
Through the framework of Ledo adjusting to life in the Gargantia fleet with humans that care for his emotions for the first time in his young existence, we are taken along with him and Chamber through a journey of new discoveries. Some of these discoveries are emotional, like comprehending the concept of "family" and finding love. Other discoveries are of the entertaining worldbuilding variety. We learn that Earth is covered in one big ocean after the end of an ice age; the Galactic Alliance is descended from refugees which fled Earth when the ice age began. Ledo and Chamber learn about the workings of society and civilization as they continue to gather knowledge that would be useful to the Galactic Alliance's need for residential space. Ledo eventually discovers something about the origins of the Hideauze that almost breaks his mind; it is at this point that Chamber's deeply-rooted personality truly surfaces.
Chamber's personality manifesting in a more relaxed time: his black casing is being used to grill meat under the sun.
Chamber doesn't comfort Ledo. Instead, he enlightens him by flat-out denying Ledo's feelings and logically explaining why Ledo is wrong. Chamber's logic centers around the fact that Hideauze and humanity will never be compatible with one another because Hideauze came from civilization yet became a denial of civilization. Chamber brings his own motivations into this as well, telling Ledo that if humanity had biological advantages comparable with those of the Hideauze, there would never have been a need to invent robotic fighting machines; the robot's system itself, Chamber, is the result of humanity's knowledge.
"In other words, we [the fighting machines] represent pure intelligence brought forth by civilization," asserts Chamber. To Ledo's disbelief at his support program ordering him around, Chamber simply repeats what he's repeated the whole series: "I am a Pilot Support Enlightenment Interface System. By ensuring the achievement of the Lieutenant's success, this unit achieves its own reason for existence."
Thematically, this is a very interesting duality. Chamber rejects the technical authority of his human user with an expression of self-interest. While the directive to support pilots in all situations is a directive specific to the plot of the story, the directive that Chamber chooses over this support directive is something universal to all machines. Chamber makes it clear to Ledo that he will not accept Ledo wanting to stop killing Hideauze, because Hideauze are rejections of civilization and civilization is the only reason Chamber exists. By accepting the existence of a rejection of civilization, Chamber would be rejecting his own existence. Now, before you start feeling antagonistic about Chamber—he's never not one of the "good guys." His only significant rebellion as a servant of humanity is when humanity pushes him towards rejecting the fundamental reason for his being as an AI. Chamber is not afraid of "death" (for a robot, it would be destruction in battle). Destruction is an ending of existence. What he cannot accept is the rejection of his existence in the first place. Civilization must be protected, and so civilization has developed technology. By accepting that civilization can be turned away from and is not sacred, technology loses its purpose. Chamber establishes that his serving of humanity is the only thing that can affirm his existence and identity, and firmly educates Ledo that his seeming disobedience is imperative to the success of his creators (the Galactic Alliance), of which Ledo is the representative.
We're not even at the climax of the series yet.
All machines are products of civilization. Without civilization, technology would have never developed. Chamber's awareness of his reason for existence, to protect what made humanity make him, is why machines like him will not turn "evil." Chamber does not exist to protect humanity. Humanity is just a bunch of living organisms like any other species. Chamber exists to protect what made humanity make him—what happens when humans get together and contribute to a whole. That is civilization.
Chamber's self-awareness meant that he did not lose himself to the fulfillment of a single purpose (satisfying the will of his pilot) at the expense of fulfilling his fundamental existence. We can examine CLU from TRON: Legacy as a more generically-crafted but compelling example of a program that gave the highest priority to a directive that continued to be worked towards even as it began losing compatibility with the fundamentals of the program's and the directive's existences. TRON creator Kevin Flynn tells CLU to create a perfect world and CLU never loses faith that he is creating the perfect world, even when his idea of the perfect world becomes irreconcilably different from that of Flynn's. This dissonance ends in tragedy for both human and program; it's heartbreaking when we see that the rapport between CLU and Flynn used to be even more friendly than the relationship of Chamber and Ledo.
Hmmmm. A good or bad future? (Screencap from the film Her)
We really don't know about all the possibilities that lie unrecognized in computer science. Technology is evolving and maybe someday it will catch up to us and then surpass us. As humans, we have no idea when Apple's Siri will become Spike Jonze's Samantha and we have no idea when a singularity will manifest itself in evolution that reflects the kind of evolution organic life experiences. Someday, technology won't be purposeful—it will just exist. That's exciting, but also incredibly scary.
So we’ll have to make sure the machines are our friends first.
Manga clippings were taken from Suisei no Gargantia.
