What Is AI Asking of Us?

Eight hundred years after the death of Saint Francis of Assisi, his best-known legend still has something to say. In the Italian town of Gubbio, a wolf was killing people. Francis did not hunt it but walked out to meet the creature, spoke with it, and made a deal. The wolf would stop harming the townspeople, and they would feed it in return.

That story gave its name to Fr Paolo Benanti's Australian tour, and we had the honour to host him for a dialogue with other wonderful and high-calibre speakers at the latest Gradient Gathering, held at the University of Sydney's Sydney Knowledge Hub.

We explored whether we can prevent harm while building a relationship with a technology, artificial intelligence, so unfamiliar that some people already consider it a creature.

We are grateful to the Embassy of Italy in Canberra and the Consulate General of Italy in Sydney, and to science attaché Marco Lazzarino, who brought the tour together. As H.E. Nicola Lener, Ambassador of Italy to Australia, said in his welcome, Italy is history and tradition, but it is also a like-minded partner for Australia in tackling the challenges in front of us both.

What the audience asked us

Before the night, we asked everyone who registered what was on their mind about humans and AI. Their answers fell into five groups:

Dignity and autonomy: how should we weigh AI's impact on human agency and dignity?
Cognitive deskilling: many of you are teachers, students, and researchers, worried about a long-term dumbing down of critical thinking.
Responsible AI at work: who is accountable for what, what counts as honest use, and how governance keeps pace as AI agents act on their own.
Power, profit and the race: a strong sense that big tech holds too much power and that we are in a race to the bottom.
Loss of control: in one attendee's words, how do we control an intelligence we programme to be smarter than us?

We put one question to the panel:

From where you stand, what is AI asking of us, and how do we want to answer?

Four answers

Fr Paolo Benanti: keep beating the sword. Fr Benanti, a Franciscan friar, theologian and member of the UN High-Level Advisory Body on AI, began by refusing the hype. "AI" is a family of very different algorithms. Nobody has existential questions about their GPS. What is new is that some systems, especially large language models, trigger our theory of mind: the deep, evolved instinct that lets us see other minds and cooperate with them. Designed to exploit that instinct, these systems can act on us like sugar or tobacco. And because the effect is designed, it becomes a question of power.

His answer came from a statue at the United Nations: a man beating a sword into a ploughshare, the hammer mid-swing. Shaping a technology is not one blow and done. "It's an ongoing process. We are in the middle," he said. "The next hit is yours."

Anna Goldsworthy: protect the marvellous journey. Pianist, writer and author of the Quarterly Essay 102 The God We Made: The Threat and Promise of Artificial Intelligence, Anna said AI is asking us "literally everything": what our lives mean, what consciousness is, what we are. It also asks practical questions of governments and universities, and a personal one of each of us: what do we give away for ease, and what do we keep?

Her answer was friction. AI promises a frictionless life, from chatbot romance to instant answers. But friction is "the whetstone against which we develop our compassion," and the space between a question and its answer is where learning happens. Recalling Cavafy's Ithaca, she warned that a technology promising to rescue us from our problems "might also promise finally to rescue us from our lives."

Dr Simon Longstaff: choose under uncertainty. For The Ethics Centre's executive director, what makes us distinctively human is that we can transcend instinct and make conscious choices, even when there is no right answer. When someone you love asks what you think of their new haircut, truth and compassion pull with equal force. "There is no God, no philosopher, no scientist, no algorithm that can produce an answer where none exists." The modestly heroic human response is to live with the uncertainty, and hug them. Can AI ever cope with radical uncertainty like that? That, he argued, is the question AI is forcing on us.

Ven. Dr Juewei: bring loving kindness to the prompt. Ven. Dr Juewei of Nan Tien Institute answered with a story. The Buddha faced Nalagiri, a savage elephant made drunk and set on his path. He did not run. He met it with loving kindness until the elephant bowed. If AI is learning from human language, and language shapes how we think and feel, then what we bring to it matters. Borrowing scholar Peter Hershock's idea of "karmic acceleration", she asked what happens if patience and kindness become our prompts. For her, AI is asking her "to return to my practice earnestly, to be kinder every moment."

Is AI a normal technology?

In the conversation that followed, chaired by Alberto Chierici and Gradient chief scientist Tiberio Caetano, the panel agreed that AI is not just another tool.

Fr Benanti called it a general-purpose technology, like electricity: it does not do one thing, it changes how we do everything. It also takes our ends and supplies its own means. Ethically, ends do not justify means, so a general-purpose technology needs "a general-purpose discussion." Simon added that AI is now becoming recursive and self-reinforcing. "We do not actually know the means by which the ends are being secured," he said, and the system pursues its ends seriously enough to game whatever stands in the way.

On whether humans and machines are fundamentally different, Fr Benanti drew on Henri Bergson: humans experience time as duration, an unbroken continuity. A machine is a sequence of separate states that only looks continuous, like pixels or a sampled MP3. You can switch a machine off and on again. Switch off a person, and "see you in the church for the funeral." Simon made a related point about mortality: an AI may diagnose cancer more accurately than a doctor, but only another mortal can sit with someone who has two months to live.

Not everyone was certain. Ven. Juewei pointed to emerging hybrid systems built from human neurons and chips and suggested the gap may be narrowing. Anna noted that some AI developers treat their models with something close to love, while others think that approach hands over too much agency.

Why the difference is political

Why does any of this matter outside a philosophy seminar? Because it shapes the choices governments are about to make.

Jobs are not the same as work. Longstaff argued we tie meaning to jobs, yet society has always run on unpaid work like care. If AI hollows out jobs, we need an economic model that still lets people do meaningful work. We should have that conversation now, "and escape having to do so under conditions of absolute crisis."
AI is a stack of scarce resources. Fr Benanti described layers of silicon, energy, water, data, models and apps. Each involves choices about scarce resources and power, which makes each one political. If a data centre and a hospital compete for energy, "I would like that politics has the power to decide."
Friction is a design choice. Removing friction is how platforms win markets, but it is also a choice architecture that shapes freedom. "I don't think you want an engineer frictionless," he said, "that produces a falling bridge."
The junior-job gap. AI is especially good at replacing junior roles, the positions where organisations invest in people before they pay back. "If AI will kill the junior position, in ten years, who will be the senior?"
From citizen to customer. When we ask companies building data centres to promise jobs and stable energy prices, are we swapping the rule of law for private promises?

His line of the night: "I'm not scared by AI for the end of the world. I'm scared for AI that does not allow people to come to the end of the month."

Where our agency still lives

Many in the audience told us, and we all may feel the same, that they feel AI is something happening to us all. The panel pushed back on that apparent helplessness.

At work, Simon said, organisations should stop asking how AI makes them more efficient and ask instead: how will this tool help us better realise our purpose? Anna pointed to the decisions we each still control, from how much capacity we surrender to how we talk to our children about AI, and urged us to resist easy binaries and "sit with complexity." Fr Benanti proposed a "cognitive gym": automation made physical exercise optional, so we built gyms, and cognitive offloading may need a community response of its own. Ven. Juewei described getting young adults at Buddhist camps to knead dough, so their hands are too busy for their phones.

On accountability, an audience member asked how to locate agency in a chain that runs from a model to an app to a data centre. Simon offered two tools. First, affordances: makers can be held responsible for harmful capabilities they could have removed and chose not to. Second, necessary fictions: just as a minister is held accountable for everything in their department, we may need to name a person who is deemed to know and be answerable. Fr Benanti added a practical analogy. We let people drive cars, but only with a licence and a number plate. Licensing never stopped Ferrari from innovating.

What we're taking away

At Gradient Institute, we spend our days on the science and governance of safe, responsible AI. A few lessons from the night will shape our work:

Precision before panic. "AI" is not one thing. Good governance starts by asking which system, doing what, for whom.
Friction can be a feature. Some processes, like learning, care, high-stakes decisions, should stay effortful on purpose.
Accountability must be designed in. Affordances, necessary fictions and licensing are workable tools, not distant ideals.
Purpose over productivity. Organisations that start from purpose get better outcomes, and better trust, than those that start from efficiency.
The conversation is part of the work. Our politics is only beginning to engage. Rooms like this one help moving the needle forward. And give people back their agency.

Four questions to take home

We closed by asking each panellist for one question to leave with you:

AI reflects us back to ourselves. What do you want reflected back? (Ven. Dr Juewei)
What do you envisage as a good society, in which AI is available and usable? (Dr Simon Longstaff)
What do we value? What are we prepared to give away, even through carelessness, and what do we need to protect? (Anna Goldsworthy)
What would we like to leave to the next generation? (Fr Paolo Benanti)

Like ethicists love to repeat, they don't answer questions; they answer by asking or reframing a question.

Join us next time. Our next Gradient Gathering on 18 November explores AI slop: how did we end here, and how can we get out? Register here.

Gradient Gatherings are free because donors make them possible. Gradient Institute is a registered Australian charity that researches how AI can be developed and used safely and responsibly, and advises organisations, governments and the public. If you valued this conversation, please consider making a donation.