35 Comments
User's avatar
Rachel Maron's avatar

Farida Khalaf isn't just diagnosing the problem; she's exposing the operating system behind it: unchecked acceleration, masked as innovation, powered by the extraction of trust.

This isn’t just about AI. It’s about who we trust with the future and why. When tech monopolies use our conversations, communities, and resources without consent, they’re not building trust; they’re burning it for fuel. AI isn’t running on math alone. It’s running on trust debt: our implicit belief that someone, somewhere, is thinking long-term. But no one is steering. Just optimizing.

Khalaf brilliantly reframes our cultural mythos: from Tom & Jerry’s celebration of clever domination to Pixar’s reverence for connection. But it’s more than narrative. It’s governance by story, and right now, the story being written is one of conquest, not stewardship.

At Trustable, we argue that trust isn’t a byproduct. It’s infrastructure. Every system that scales without permission, without community input, environmental accounting, or equitable design, erodes the very foundation it depends on.

This piece is a wake-up call. If we don't make trust measurable, operational, and accountable, then we will keep cheering for clever systems while they quietly strip us of our sovereignty, resource by resource, until we have nothing left but awe and no agency.

It’s not too late to flip the script. However, that requires us to stop thinking of trust as a soft sentiment and start treating it like the hard constraint it actually is.

Alex Pardo's avatar

AI is the “bitcoin” of this decade

Farida Khalaf's avatar

the euphoria is identical

Farida Khalaf's avatar

kinda agree with you

Claire Fratson's avatar

Wow. Thank you Farida. This is very educational for me. The thing is I know I don't like or trust the AI movement, but I find it difficult to express why. Its a gut feeling that something is very 'off' and out of control. Thank you for writing such a thought provoking, informative piece. I keep reminding people old enough to remember the film Terminator....can't help but feel the human race is actively creating its own demise 🙏

Farida Khalaf's avatar

I do believe that Human will take control and turn the table or lets say I pray for that

Diana's avatar

I am joining the conversation little bit later. I am really interested did you ran this post with any LLM model and what was the feedback? Did the machine had anything to say about tour concerns? Because, sadly, I use them for grammar check etc, english is not my native language, and at one point I wrote something on this exact topic, significantly wicker then this post. But when i run it with Claude he had much more to say, not just grammar corrections :)

Bran Knowles's avatar

Thank you so much for this, for being brave enough to call it what it is. The environmental costs are too often glossed over. When I see people talking about all the fun, new ways they’ve integrated AI into their practices (using it to proofread for them, for example), I just shudder at the associated environmental costs and think, “Oh great, there’s no unplugging that now.” Once people find a use with these tools, it gets harder to imagine the world you’re describing. Do you see a way of unwinding? Of pulling back? The most depressing moment for me was when I learned that the US government was using OpenAI for some of its military stuff (details escape me). I thought, “That’s it. We’re wedded to this tech now.” What will it take, do you think, for us to say, “Enough is enough. We can’t afford this AI, this way”?

Farida Khalaf's avatar

You're right to feel concerned, once AI becomes useful, it's hard to imagine stepping back. But change is inevitable. What matters is how we tame it, so it serves us instead of consuming us. We need regulation, inclusion, fairness, and most importantly, limits. We have to hit the brakes on runaway capitalism long enough to measure the damage and offset it. The good news is: we don’t need to unwind everything to make meaningful change. Even if we can't unplug AI entirely, we can slow the rate, shape the governance, and redirect the incentives. For example:

* A global moratorium on new data centers in water-stressed regions

* Public transparency laws for AI infrastructure resource use

* Binding international agreements on AI in military applications

* Local ownership models for AI tools that serve community needs

This isn’t about stopping progress, it’s about making sure it doesn’t steamroll everything in its path.

Bran Knowles's avatar

Yes, and on reflection, I think it's probably important to frame it not as "Can we stop/unplug/unwind?", but "Can we move on to something better?", the latter being much more energising. And key to being able to do this is recognising that there are natural limits to innovation (a climate/water/pollution budget), BUT that constraint has always been a powerful agent of the imagination, giving shape to the space we need to construct our best ideas.

Bran Knowles's avatar

Also, at the risk of engaging in gross self-promotion (eek!), last week I wrote a post about distrust of AI as being a symptom of the frustrated strivings of the ego… the hope that our technology futures could be other, and despair that they are not, and that it being other feels so impossible given our current techno-politics. It seems to resonate with your Pixar analogy. If interested, the post is here. https://open.substack.com/pub/trustbranknowles/p/5-lessons-of-the-doppelganger?r=5z1pkv&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false

I’d love to hear your thoughts on that as well.

Farida Khalaf's avatar

This is a powerful piece, thank you for sharing it. The connection you draw between distrust and the frustrated strivings of the ego really resonated. It echoes exactly what I was circling with the Pixar metaphor: the hunger for a more humane, connected, and chosen technological future, rather than one we’re forced into by default.

Your reflections on “trust friction” and the danger of pipiking trust itself are especially sharp. That shift from trust as something earned to trust as something assumed or marketed, is quietly corrosive, and exactly the kind of sleight of hand we need to call out. We’re clearly wrestling with the same tension: the grief for futures foreclosed, and the stubborn hope that imagination still matters.

Bran Knowles's avatar

Great! I'm so pleased it resonated! I guess the million dollar question is how to harness that shared hunger for something more, how to collect it up and blast it so that it promotes real change in the way technology relations are configured. I think enough are feeling this way, but few feel that they have agency.

Clinton's avatar

Farida, this is not just journalism; it’s cartography for our moral imagination. You don’t merely warn us about AI’s extractive future; you map the terrains of complicity, narrative distortion, and infrastructural erosion with scalpel-like precision.

Your invocation of the Koch snowflake is a stroke of conceptual elegance; what begins as “beautiful mathematics” soon reveals its necropolitical underside. It’s not just that LLMs grow infinitely; it’s that they do so on borrowed time, borrowed water, and borrowed agency. We are not just spectators to this recursive inflation of complexity; we are resources in its metabolism.

What makes your argument so potent is that it refuses to collapse into dystopian fatalism. Instead, you expose the manufactured urgency driving this race - a narrative sleight of hand masquerading as inevitability. It’s not a singularity we are approaching, but a supply chain rupture with ideological roots: cognitive monopolies wrapped in startup mythologies.

You’re right to locate this in culture. The “Tom & Jerry” metaphor, far from playful nostalgia, becomes a devastating parable of how cleverness has been systematically decoupled from care. “Cheer for Jerry, no matter the cost” - it’s not just a media message, it’s a political theology. A worldview.

The planetary ledger you present (of water, land, energy, and dignity) should be read not as alarmism, but as audit. The pixelated mirage of “innovation” is collapsing into a global thermodynamic reckoning. You have shown us, compellingly, that the infrastructure of ‘thinking machines’ is being built atop the crumbling foundations of living systems.

And your conclusion is the hardest truth: once locked in, course correction becomes exponentially harder. Which means now, not tomorrow, is the moment of moral calibration.

We must not merely ask what AI can do. We must ask: Who benefits? Who pays? Who decides? And perhaps most importantly: who is still allowed to ask questions at all?

Thank you for giving us permission to ask again. And to ask better!

Farida Khalaf's avatar

Wow , your words truly stopped me in my tracks. I can’t thank you enough for taking the time to read so deeply and respond with such clarity and care.

You’ve articulated exactly what I hoped the piece might evoke, not just concern about AI's impact, but a reorientation of how we think about impact itself: not in abstract fears, but in real, material costs and cultural myths. Your phrase “cartography for our moral imagination” is something I’ll carry with me, what a beautiful way to describe the work we’re all trying to do in this moment.

And you’re absolutely right: the issue isn’t inevitability, it’s narrative control. The metaphors we normalize shape the infrastructures we tolerate. Thank you for naming that so powerfully.

More than anything, your response reminds me that this isn’t a solo voice in a void, it’s part of a collective questioning. Thank you for joining in, and for asking better.

Erik's avatar

A bottle of water per query? Do you have a source for that?

The best I could find was a paper from 2023 that states 500ml for 10-50 medium length queries = ~20 milliliter (4 US teaspoons) per query.

https://arxiv.org/html/2304.03271v5

Apparently (although I could not find a source for that) equivalent to 3 seconds of TV watching.

Farida Khalaf's avatar

Great question Erik, and you're absolutely right to dig into the numbers. That “bottle of water per query” stat is more symbolic than precise, but it’s backed by real concerns. The Washington Post (Sept 2024) reported that a single 100-word GPT-4 reply can use around 500 mL of water and 0.14 kWh of electricity, depending on the cooling system and location.

So while the exact number can vary, the bigger point stands: at scale, these systems have a real environmental impact. I appreciate you bringing solid sources into the conversation—helps keep it grounded!

Erik's avatar

Well let me put it this way: While I’m probably more positive about the impact of AI than you are, I was interested in your article to keep reading both sides. However, by using “symbolic numbers” to inflate the actual data by a hundred fold, you lost me in your arguments.

To come back to the facts: The Washington post article is paywalled, but this article dissects all the claims, and comes to the same conclusion as I do: 500ml per query is grossly incorrect when going back to the original research.

https://www.seangoedecke.com/water-impact-of-ai/

Farida Khalaf's avatar

You were right about the water consumption figure - I've corrected it. Thank you for the fact-check.

Farida Khalaf's avatar

Thanks for the fact-check - and honestly, I'm disappointed to lose a thoughtful reader over imprecise phrasing.

You're absolutely right about the 500ml figure - that's specifically for longer tasks like 100-word emails, not simple queries. The research actually shows "every 20-50 ChatGPT queries consumes roughly one bottle of water."

I genuinely try to research these claims thoroughly, but your correction is spot-on. I'll revise that section based on your data and give you credit for the fact-check - this topic is too important for sloppy framing to undermine the broader conversation.

I really do want to get this right. What's your take on the infrastructure sustainability questions, setting the specific consumption figures aside?

Erik's avatar

Thanks for the quick reply. I had a look at these articles but the only one that specifically states a number per queries is the second one, which points to the same paper that I mentioned: “estimated to consume 500 ml of water per 10-50 responses..”

I think you may have missed that second part?

Farida Khalaf's avatar

Totally fair Erik, and I really appreciate your attention to detail. I did read those reports, but deliberately didn’t lean into exact figures because this article wasn’t meant to be academic, it’s more of a socioeconomic reflection than a technical breakdown.

The “bottle of water per query” line was used figuratively, to help illustrate the real but often hidden costs of AI infrastructure in a way that sticks with readers. That said, I completely agree: when we use analogies, clarity matters, and you’re right to call it out.

Thanks again for engaging so thoughtfully, this is exactly the kind of conversation I hoped the piece would spark.

Tessina Grant Moloney's avatar

Interesting read. I'm with you on the big picture, the mission, and most of the details. However, go easy on the bolding. When so much of the article is emphasized with bolding that looks exactly like the header format used, it's hard to stay focused or to identify what's truly most important. 😅

I do love the snowflake symbolism used and agree wholeheartedly that the planet isn't ready for AI at scale. Lots of people are worried about AGI/ASI, but I think even current LLM will pose an existentialist risk even if they never gain another ounce of capability. The mass consumption coupled with scaling biases across basic needs (like housing, hiring, health, etc.) is unsustainable.

Farida Khalaf's avatar

Thank you so much for reading and for the thoughtful comment, I really appreciate it. You're absolutely right about the bolding, and I take your note to heart. I’ve been leaning on it a bit heavily, partly because I haven’t been using visuals or graphs to break things up and wanted to avoid putting readers in “passenger mode.” But you're right—too much emphasis ends up diluting the emphasis. 😅 I’ll definitely rethink the balance moving forward.

I’m also really glad the snowflake symbolism resonated with you, and I completely agree LLMs at current capability levels already raise deep structural risks, especially when deployed at scale without adequate checks. The examples you mentioned, housing, hiring, health are exactly where it can get messy fast.

Also, I noticed you’re a PHD student, that’s awesome. I’d love to hear more about what you’re working on. Hope to make it to Dublin one day soon, maybe even get the chance to say hi in person!

Tessina Grant Moloney's avatar

Hi Farida! Thanks for responding.

I was a content product manager, and now I do QA work to audit content before publishing. So I do pay attention to content design quite a bit. We had some authors who did a lot of over-bolding that I had to fix every week, so the memory runs deep 😅. That said, I'm sure I completely bulldoze my way through other content design best practices that UX/UI people would do better.

If you ever make it to Madrid, let me know! That's where I'm based. My school is in Ireland, but GCAS is 100% remote, so we're dispersed. I am going to Ireland next week to meet up with schoolmates for some research, but this is totally self-organized and voluntary. We want to design a method to examine the extent to which current popular hiring AI tools show bias for/against certain groups.

My PhD research is related to this, but it's a little more broad. It examines the extent to which AI and automation overall has contributed to job insecurity among women and marginalized persons. Here's a link to a recent post that basically describes that I'm studying: https://thecoldboot.substack.com/p/examining-ai-driven-job-displacement

I hope you subscribe! I'll be sharing my insights here along the way. 🙃

Farida Khalaf's avatar

I’m really interested in your research and will definitely be looking forward to reading your thesis someday soon. And yes, I’d be happy to meet up in Madrid whenever that happens! Thanks so much for your kind note, and you’re always welcome to audit my content anytime you feel like it. 😊

Tessina Grant Moloney's avatar

You're welcome, and thank you! I was away in Ireland doing AI research with a friend, so I'm just catching up on messages. The dissertation is definitely underway. 🥳 I don't think you follow me or subscribe to my stack, so I'm not sure you'll encounter my work but it was nice chatting. Wishing you all the best with your writing as well. 😊

Jean-Paul Paoli's avatar

The Snow image is beautiful… and clearly Ai is ressources heavy. But To me Tom & Jerry story is over-interpreted they are friend after alll… it’s not a story about speed , it’s a simple story where small (kids) can outsmart the mighty (adults) … same story as David and Goliath for instance

Farida Khalaf's avatar

Thanks for sharing your perspective! I see where you’re coming from, but I’d say Tom & Jerry were never really friends. Jerry was more like an uninvited intruder, not the adored, chosen pet. The dynamic is one of conflict, not companionship.

As for David and Goliath, I think it’s a different story altogether. David’s victory wasn’t luck or just outsmarting by chance. He studied his enemy carefully, prepared thoroughly, and had a clear plan and mission. His success came from faith, skill, and precision, hitting the giant’s weakest spot with full confidence and readiness.

So to me, David’s story is about preparation, belief, and sharpening your skills, not just the underdog getting lucky. it is a reminder that when we harness our unique strengths with intention and faith, amazing things can happen. It’s not about luck, but about purpose and dedication.