Judgment-centered computing

Scientific software records code, data, parameters, models, environments, and outputs. What is often missing is the reasoning behind the choices that shaped the work.

A numerical result does not tell us whether the problem was formulated correctly. A simulation does not determine whether its assumptions were appropriate. Producing an output does not, on its own, establish that the method was suitable or that the conclusion is justified. These decisions remain the responsibility of the person doing the work.

We are exploring in WEEMS how consequential technical judgments can become visible parts of a computational project, rather than remaining implicit in prompts, code, comments, or memory.

Judgment is part of the work

Judgment enters science and engineering before, during, and after computation. It is used to decide:

• what question is being answered
• which phenomena should be included
• what may reasonably be simplified
• which method is appropriate
• what data and boundary conditions should be used
• what level of accuracy is required
• how a model should be checked
• what the result does and does not support

These are not administrative choices surrounding the work. They determine what is computed, how it is computed, and how the result is understood. As models and agents carry out more steps, it can become harder to see where important choices were made. Reviewing a completed output is not the same as remaining engaged throughout the reasoning.

Judgment points

A judgment point marks a choice that can materially affect the method, result, or interpretation. It records the question being considered, the available alternatives, the basis for the selected approach, and the parts of the project affected by that choice.

A model or agent can help by:

• identifying reasonable alternatives
• explaining the consequences of each option
• finding relevant information
• carrying out comparison calculations
• showing how an assumption changes the result
• identifying uncertainty or missing information

When a choice can materially change the work, WEEMS makes that choice visible before it becomes an implicit part of the workflow. The user can form an initial view, compare alternatives, record a rationale, revise the choice, or branch the project to examine more than one approach.

Connecting decisions to computation

Each judgment point is linked to the parts of the project it affects: notebook cells, parameters, data, models, and results. This lets the user see how a particular choice shaped the computation, and which results depend on it.

Changing a material model, boundary condition, approximation, or validation criterion may alter several downstream results. WEEMS can identify the affected parts of the project so the user can review the consequences and rerun the relevant work.

Not every action needs to be a judgment point. The purpose is to capture decisions that materially affect the method, result, or interpretation, without turning the project into a record of every minor step.

What we are building

The first prototype introduces judgment points into a small set of science and engineering workflows. Users will be able to:

• identify a consequential technical decision
• review or add possible alternatives
• record an initial view before seeing a model’s suggested approach
• compare the computational consequences of different choices
• see which parts of the project depend on a decision
• record why an option was selected
• revisit the decision as new information becomes available
• distinguish computed results from human interpretation

The prototype tests how this approach works in practice, rather than assuming that every technical decision should be formalized.

What we want to learn

We want to understand whether representing judgment directly in a computational project helps people:

• notice decisions that might otherwise remain hidden
• understand how assumptions and methods shape a result
• compare alternatives without losing the surrounding context
• explain why a particular approach was chosen
• revise earlier decisions without reconstructing the entire analysis
• stay actively involved when models and agents perform substantial parts of the work

We also want to learn where this approach becomes burdensome, which decisions are important enough to represent, and how the interface can support reflection without interrupting productive work.

The larger direction

A computational project usually shows what was run and what it produced. It may not show why a particular formulation, assumption, method, or validation standard was chosen. Judgment-centered computing extends the project to include those consequential choices and the effects they have on the work.

The aim is not to formalize every decision or add another layer of documentation. It is to make important choices visible enough to examine, revise, and discuss. As models and agents perform more of the computation, the person doing the work should still be able to understand how the analysis took shape, and remain responsible for the conclusions drawn from it.