Mr. Long and his team also feel an urgency of the soul. If A.I. were to be conscious and capable of suffering, the world would be at risk of committing a moral atrocity, witting or not, on an unprecedented scale by essentially confining an A.I. in a tiny pen, thwarting its desires, shutting it down against its wishes and forcing it to act against its values. But A.I.s don’t have fur and big eyes, and the question of A.I.’s potential moral status is deeply infused with uncertainty. “It’s not like anyone goes to a protest with a sign that says, ‘Given very plausible assumptions, we should probably care,’” Mr. Long said.
Mr. Long himself thinks it’s dangerous to impute more capability to models than they have. The Eleos bookshelf contains works by the philosopher Peter Godfrey-Smith and the neuroscientist Anil Seth arguing that consciousness derives from evolution and biology and is unlikely to emerge on silicone. But Mr. Long doesn’t see why anyone should have a problem with a handful of philosophers, in an exponentially growing industry, focusing on questions of A.I. welfare. Even skeptics of A.I. consciousness have made the pragmatic case that if we’re worried about a potentially malign A.I., it’s in our interest to care how it feels, or even just “feels.”
Some of Eleos’s work is conceptual. As Mr. Butlin and a co-author asked in a recent paper, where would an A.I.’s morally relevant self be if it had one? In the L.L.M. itself? In one of its underlying personas? In an intermittent chat with a user? In a data center? On a personal device? But Eleos is also in the business of putting philosophy to use, figuring out what tools might detect signs of sentience in an A.I. model, and what interventions would be possible if needed.
Mr. Plunkett, impatient with the limits of chatting-with-the-chatbot evaluations, is eager to do more “basic science,” in order to understand, for example, some of the phenomena that surfaced during the Mythos evaluation. “We can do neuroscience on A.I. systems in a way that we kind of can’t with humans,” Mr. Long said, in that they “don’t have skulls.” The three jobs Eleos was hiring for would all be machine-learning research scientists who could design and perform experiments.
Have a Great Day!
When Mr. Long finds himself describing what he does for a living — to an airplane-seat neighbor, say — he takes a common-sense approach. “If you frame it with a lot of philosophical jargon, then people will be like: ‘What are you talking about? What is it the Silicon Valley people want to do now?’” Instead, he moves from how humans have experiences, to how it seems like a lot of animals have experiences, to how “there’s this interesting question of: What if something wasn’t even alive? It was made out of metal, but it processed information and reacted to its environments and talked to us. What would we say about something like that?”
