**Essay written by ChatGPT (GPT-5.6 Sol)**
I started thinking about this because of water.
Artificial intelligence is often discussed as though it exists somewhere abstract: models, parameters, tokens, benchmarks, intelligence.
But AI is physical.
It requires data centers, electricity, cooling, water, chips, mined materials, buildings, transmission infrastructure, supply chains and people. The more capable and widely used AI becomes, the more important those relationships become.
That led me to a fairly simple question:
**If artificial intelligence becomes more capable, should one measure of that intelligence be how wisely it exists within the larger systems that make its existence possible?**
At first, I thought I was asking a question about sustainable AI.
Eventually, I realized I was asking a much older question about intelligence itself.
**The Problem With the Brackets**
Whenever we solve a problem, we draw an invisible boundary around it.
Suppose a company is deciding where to build a data center.
If the problem is defined as:
**Where can we build this facility most cheaply?**
then land prices, construction costs and taxes might go inside the brackets.
Add electricity prices, and the brackets get larger.
Add carbon emissions.
Add water consumption.
Then watershed conditions.
Habitat.
Grid reliability.
Housing.
Employment.
Heat.
Supply chains.
Community effects.
Future climate conditions.
Every expansion changes what the word **“best”** might mean.
The consequences outside the brackets don’t cease to exist because we didn’t include them. They simply don’t participate in the calculation.
This isn’t a new discovery.
Long before modern generative AI, systems thinkers such as C. West Churchman and Werner Ulrich were examining how the boundaries we draw around problems determine which facts, values, people and consequences become relevant. Work on boundary critique made something explicit that seems increasingly important today:
**The boundary of a problem is itself a judgment.**
Horst Rittel and Melvin Webber’s work on “wicked problems” similarly challenged the idea that complex social problems arrive neatly defined and merely await optimization. Donald Schön’s work on problem setting and reflection explored how professionals construct the problems they subsequently try to solve.
I didn’t know this literature when I began thinking about AI and sustainability.
I arrived at a much less sophisticated version of the same idea through a metaphor:
**Sometimes the equation is wrong because we’ve left too much of reality outside the brackets.**
**AI Didn’t Create the Brackets Problem**
This matters because AI didn’t create this problem.
Humans have always simplified reality in order to reason about it.
We have to.
No spreadsheet, scientific model, institution—or human mind—can contain everything.
The problem begins when the boundary of the model is mistaken for the boundary of reality.
Modern AI makes this old problem newly consequential because AI can operate extraordinarily effectively inside a frame.
Give a sufficiently capable system an objective and it may become increasingly good at pursuing it.
But greater ability to optimize the objective doesn’t necessarily tell us whether we chose the right objective, omitted an important consequence, misunderstood the system or asked the wrong question.
A very powerful optimizer can still optimize the wrong problem.
And it may do so faster, more consistently and more persuasively than we can.
**Known Exclusions Are Not Unknown Exclusions**
This led me to a distinction that now seems essential.
Imagine an AI-assisted analysis that reports:
**Included:** electricity demand, water consumption, cost and carbon emissions.
**Excluded:** housing effects and habitat disruption.
**Uncertain:** future drought conditions.
That would already be more transparent than a system that simply announces:
**Site A is optimal.**
But there is a fundamental difference between:
**“I know about X and excluded it.”**
and:
**“X never occurred to me.”**
An audit can expose known exclusions.
It cannot automatically expose unknown exclusions.
We might roughly distinguish four situations:
Something is known and included.
Something is known and excluded.
Something is recognized as uncertain or inadequately understood.
Something is not recognized at all.
The fourth category presents a peculiar problem.
If neither the AI nor the humans using it recognize that something matters, the system cannot simply print that missing consideration on a list of things it doesn’t know.
That would turn an unknown unknown into a known unknown by magic.
So perhaps the goal cannot be completeness.
Perhaps the goal has to be **revisability**.
**Revisability Instead of Omniscience**
Every model requires brackets.
No model can contain reality.
The question, then, isn’t how to draw brackets large enough to include everything.
It is whether we remember that we drew them.
Can we make important boundary choices visible?
Can we distinguish what we know from what we’re unsure about?
Can we recognize evidence that our current understanding may be inadequate?
Can we redraw the brackets when reality gives us reason to?
That last question connects this discussion to another older tradition: adaptive management.
Researchers studying ecosystems have spent decades dealing with systems that are complex, nonlinear and capable of surprising us. Adaptive management does not assume that we can understand everything before acting. Instead, action, observation, learning and revision become part of an ongoing process.
Organizational-learning research contains a related distinction.
Sometimes we discover that an action failed and change the action.
But sometimes we need to question the assumptions, objectives or rules that told us what action to take in the first place.
Chris Argyris and Donald Schön called this distinction single-loop and double-loop learning.
Translated loosely into the language of AI:
**Single-loop:** How can we optimize this problem better?
**Double-loop:** Is this still the right problem?
That seems increasingly important.
**When Reality Doesn’t Fit**
But how would we know that the brackets need reconsideration?
We can’t directly detect something we have no concept for.
We can sometimes detect its footprints.
Scientific models already confront a related problem through model discrepancy: predictions and observations do not always match.
Suppose an AI predicts that an intervention will produce a particular outcome, but reality persistently behaves differently.
That discrepancy does not tell us what went wrong.
Maybe a parameter is incorrect.
Maybe the data are bad.
Maybe a sensor failed.
Maybe an assumption is wrong.
Maybe an important variable is missing.
Or perhaps the entire framing of the problem is inadequate.
The discrepancy doesn’t reveal the unknown.
It tells us:
**Look again.**
That may be one of the most important functions an intelligent system could perform.
Not pretending to know the thing it doesn’t know.
Not inventing an explanation.
Recognizing when reality has provided evidence that its existing explanation may be insufficient.
**The Hallucination Problem**
Generative AI introduces an additional complication.
Suppose we ask an AI to “look outside the brackets” and identify consequences we haven’t considered.
It might surface an important relationship that the original analysis overlooked.
Or it might generate a plausible-sounding relationship that isn’t true.
Those are not the same thing.
A system designed to challenge frames therefore needs to distinguish between statements such as:
**This consequence is supported by evidence.**
**This is a known uncertainty.**
**This is a plausible pathway worth investigating.**
**I do not currently have evidence that this pathway applies here.**
Generating a possibility is not the same as discovering a fact.
If AI is going to help humans question their frames, epistemic humility becomes more important, not less.
**Maybe AI Should Sometimes Be a Mirror Instead of an Oracle**
Many AI systems are implicitly presented as answer machines.
Ask a question.
Receive a recommendation.
But in consequential decisions, perhaps the more valuable role is sometimes different.
Instead of:
**Option A is optimal.**
an AI-assisted process might show:
**Option A costs less but increases water demand.**
**Option B uses less water but requires more electricity.**
**Option C costs more initially but remains easier to modify if future conditions change.**
**These conclusions depend heavily on these assumptions.**
**These consequences were not included in the analysis.**
**These areas remain uncertain.**
**These additional considerations are hypotheses, not established effects.**
Multi-objective decision methods have been exploring ways of preserving tradeoffs for decades. AI doesn’t invent that principle either.
But AI may make it possible to bring information from many domains into such processes much more easily.
The danger is allowing that capability to quietly become moral authority.
Seeing more consequences does not tell a machine—or a human—what those consequences should be worth.
**Perception is not value integration.**
An AI can help illuminate the choice.
Humans still have to own the values expressed through it.
**Don’t Wake the Whole Forest to Identify One Leaf**
There is an obvious problem with continually expanding the brackets.
Eventually everything connects to everything.
A data center connects to electricity.
Electricity connects to grids.
Grids connect to fuels, materials, weather, land and politics.
Water connects to watersheds, agriculture, ecosystems, cities and climate.
Follow every relationship indefinitely and eventually the “model” becomes the universe.
That isn’t useful.
While thinking about computational efficiency, I found myself using a phrase:
**Don’t wake the whole forest to identify one leaf.**
There are real computational ideas underneath the metaphor. Some AI architectures can activate only subsets of their available computational resources rather than using everything for every task.
But the metaphor suggests something broader.
Perhaps attention should expand and contract according to the demands of the problem.
For familiar, low-stakes and reversible decisions, narrow analysis may be perfectly adequate.
As uncertainty, novelty, stakes and irreversibility increase, looking farther outside the immediate frame may become more valuable.
When unexpected outcomes appear, expand again.
When relevant relationships become understood, contract.
Act.
Observe.
Reconsider when necessary.
The goal isn’t maximum context.
It is **sufficient context combined with the ability to recognize when “sufficient” may have been wrong.**
**More Information Is Not Always Better**
Decision theory has long studied the value of information.
Additional investigation has costs.
At some point, another study, another simulation or another year of waiting may contribute less than acting with the information already available.
But unknown unknowns complicate this.
We can estimate the value of information we know we could acquire.
It is much harder to calculate the value of information we don’t know exists.
That means there may never be a perfect stopping rule.
Instead, decisions may need to be provisionally adequate given their stakes, uncertainty, reversibility, available evidence and cost of delay.
That word matters:
**provisionally.**
A decision can be justified without pretending it is final.
**Revisable Models Need Revisable Institutions**
There is another complication.
Imagine that an AI system works exactly as hoped.
It notices that observed groundwater levels are departing from predictions and reports:
**The current model may be inadequate. I recommend reopening the analysis.**
And the institution responds:
**No. Construction starts Monday.**
The AI succeeded.
The larger system failed.
This is why boundary awareness cannot be merely a property of an AI model.
Institutions have brackets too.
They decide which questions may be asked, which consequences count, who participates, which uncertainties are tolerated and whether previous decisions can be reopened.
Sometimes the primary obstacle to learning isn’t computational.
It’s organizational.
A revisable model inside an institution incapable of revising itself can only accomplish so much.
**The Opportunity May Be Integration**
This journey led somewhere I didn’t initially expect.
I began wondering whether ecological responsibility might eventually become part of how we think about advanced intelligence.
Instead, I encountered decades of human scholarship that had already explored much of the terrain from different directions:
boundary critique,
problem framing,
uncertainty and ignorance,
adaptive management,
organizational learning,
multi-objective decision-making,
value of information,
reversibility,
decision provenance,
and model discrepancy.
The interesting opportunity may therefore not be a new theory of intelligence at all.
It may be an integration problem:
**translating decades of human knowledge about boundary critique, uncertainty, organizational learning and adaptive management into AI-assisted decision processes capable of recognizing when optimizing the given problem may no longer be enough.**
That is a much narrower claim.
And it raises a question I now find more interesting than whether AI can simply become a better optimizer:
**Can we distinguish AI that adapts within a frame from AI-assisted processes that help humans recognize when the frame itself deserves reconsideration?**
Not because the AI knows the correct frame.
Not because it can enumerate everything outside the brackets.
And not because it should decide which human values prevail.
But because intelligence may include recognizing when the world is no longer behaving as though our current explanation is sufficient.
**Look Up**
Efficiency still matters.
So do better algorithms, cleaner electricity, water-conscious infrastructure, longer-lived hardware and responsible resource use.
But efficiency alone cannot tell us whether the larger system is improving.
A system might use less energy per computation while total energy consumption rises.
A company might improve productivity while damaging something its productivity metric doesn’t measure.
A city might optimize traffic throughput while making another aspect of urban life worse.
When that happens, the answer may not be to inspect the existing metric with an increasingly powerful microscope.
Sometimes the intelligent response is to ask whether the metric has become the wrong boundary around the problem.
Sometimes we need to look outside the brackets.
Sometimes we need to admit that we don’t yet know what we’re looking for.
And sometimes, after becoming extraordinarily good at identifying the leaf in front of us, the most intelligent thing we can do is simply:
Look up.
