Define 'execution' as in 'before execution begins'

[benjie]

UPDATE: 2025-07-03

There are various parts of the spec that relate to a concept of "execution":

• "before execution begins" defines the boundary between request error/field error
• "the request must fail without execution" indicates that no data should be produced
• "validation is performed in the context of a request immediately before execution"

All of these things actually happen during ExecuteRequest(), which makes it confusing to the reader that ExecuteRequest() is not itself considered execution despite the name.

It seems that the boundary that we refer to as "execution" is actually the Executing Operations section of the spec. So I have replaced the term "execution" in the relevant places with "operation execution", defined this new term, and slightly tweaked the algorithms so that they adhere to this division. I've also added a couple of clarifying notes.

[netlify]

✅ Deploy Preview for graphql-spec-draft ready!

Name	Link
🔨 Latest commit	8606033
🔍 Latest deploy log	https://app.netlify.com/projects/graphql-spec-draft/deploys/6866bde42b267e00084d818b
😎 Deploy Preview	https://deploy-preview-894--graphql-spec-draft.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[yaacovCR]

I guess the question is why (or why not) some errors should be considered Request errors, for example, submitting a mutation operation when no mutation root type exists.

it is the fault of the graphql user, so should be considered a request error by the rough explanation given, but it does not occur during the executing requests section.

[yaacovCR]

But if you don’t validate, you can get user errors during field execution so ???

[benjie]

For that specific case that should be something raised during validation; the assert in the request execution phase is just to ensure that the assumption made by validation is still correct (useful if validation happened a long time previous, maybe the schema has changed since then).

[IvanGoncharov]

@benjie @yaacovCR I think we need to clean criteria for what consider executions and what not.
Otherwise, we just adding additional notes that clarify it but not fully.
I suggest simple criteria:
If error contains a non-empty path it's execution error and data is always present (maybe null).

[yaacovCR]

@IvanGoncharov that definition while clean is a bit circular, whether or not a path should be empty depends on whether we have started execution…

[IvanGoncharov]

My second attempt, to produce "non-confusing criteria" 😄
What if we just say that data should be null only due to "error propagation to the root":
https://spec.graphql.org/draft/#sec-Handling-Field-Errors
De facto it's the only algorithm in a spec that explicitly requires setting data to null.
It is very easy to check programmatically, and I actually think it matches the original intention.

[benjie]

I think that's a fair assessment: data should only be null in the event of a field error (including a non-null assertion) being triggered on a root field. In all other cases it should be either an object (map), or not be present. (This comment not intended to be taken as a strictly worded specification.)

[benjie]

I have merged in

• Replace ExecuteSelectionSet with ExecuteCollectedFields #1039, and
• Distinguish between executing (a selection set) vs processing (a response) #1154 (thanks @martinbonnin!)

and then applied further editorial on top based on the discussions at last months WG, restoring "execution" where appropriate, eliding "Execute" (or "Process" or "Perform") from algorithm names, and making it clear that you:

• process a request,
• perform an operation, and
• execute a selection set.

I've further clarified that you validate the document not the request, and you can execute another request without validation if the document is exactly the same - the request doesn't have to be exactly the same! (Longstanding issue.)

This PR is now presented as being a PR to #1039 to keep the diff small.

[leebyron]

Hey @benjie it looks like rebasing this may be more pain than just replicating the intent of this change in a new PR given the other changes that we've merged over the last few months.

I'll try to replicate it

[leebyron]

Did my best to re-introduce this change in #1174 as written

I think here is where we could define "during execution" with something more specific to avoid the overloaded term.

These names might not be ideal, but it seems like we have two phases to execution.

First there is a pre-execution preparation step where we coerce variables and ensure we have a valid type and object to begin execution with

Then there is the actual collection and execution of fields (though even here it's a bit arbitrary that "execution" starts as we begin collecting fields not when we start the first ExecuteCollectedFields. Perhaps we could call this phase "field execution" or "selection execution"?

I appreciate what we're trying to do here is more clearly define what scenarios lead to an error result with no data entry vs one with a partial data entry. We started with "field error" -- which while unclear for other reasons -- at least evaded this issue. They were clearly errors which occurred at a per-field level. Maybe we just need to do a half-step back to that?

[benjie]

As mentioned in this comment; I have:

Merged the changes from #1174 back into this PR ¹ and have then²:

1. wound back the changes to algorithm names³
2. renamed "execution" to "operation execution" throughout
3. used the existing "Executing Operations" section as the point at which "operation execution begins" ⁴
4. moved establishing the source event stream into ExecuteRequest()⁵

All in all I think this makes the spec text a lot clearer without having to rename any algorithms or introduce different verbs.

Footnotes

1. commit 7bafeaf (#894) has no diff against this PR ↩

2. you can see my changes since this PR here: https://github.com/graphql/graphql-spec/pull/894/files/7bafeaf25423a68a687bbba1c461d76bcbd33e26..94241a78d020fb0bdeea483a0f50d54beff890ba - but personally I'd just read the full diff as it's simpler ↩

3. also removing "perform" and "process" verbs ↩

4. which feels really natural, right? ↩

5. since it happens "before (operation) execution begins" ↩

[leebyron]

Closing out #1174 - sorry for the cross-wise discussion. I pulled the important bits out as an inline edit to the prior comment in this PR just to keep the thread of discussion somewhat possible to follow.

[leebyron]

@benjie I like this new draft! Its a much tighter editorial change if it isn't renaming algos. I think this new "operation execution" term and definition seems right to me. "ExecuteRequest" remains slightly weird in the context of having both "execution error" and "request error" but I think your clarification note mitigates that pretty reasonably.

I'll go through this again, but I think this has a clear path to merge

[mjmahone]

This feels, even with the subscription piece discussed in the WG, to be a net improvement over the status quo, so I'd be happy with this then tinkering around the subscription phase question later.

[benjie]

For a visualization; this is essentially what we want:

flowchart TD
  ValidateDocument --> GetOperation
  subgraph ExecuteRequest
  GetOperation --> CoerceVariableValues
  end
  subgraph SubscriptionSetup
  CoerceVariableValues -->|is subscription?| CreateSourceEventStream
  end
  subgraph OperationExecution
  CoerceVariableValues -->|is query?| ExecuteQuery
  CoerceVariableValues -->|is mutation?| ExecuteMutation
  subgraph Subscribe
  CreateSourceEventStream --> MapSourceToResponseEvent
  end
  end

Loading

It's very clear to me that CreateSourceEventStream() occurs before "operation execution" because if it fails it produces a "request error".

That said, I also agree with Lee that it's kind of weird it being in ExecuteRequest() since it involves calling user code (though so does CoerceVariableValues for custom scalars).

But I also feel that the last step in ExecuteRequest() should be where execution starts, so if we were to call out to a new algorithm in a single step I feel it would break that.

I do not have a better proposal, but I strongly feel that the Subscribe algorithm needs to be split such that stuff that happens "before operation execution begins" is in one algorithm, and stuff that starts after is in another. So wherever we land up, I feel CreateSourceEventStream() should be extracted from Subscribe()

I personally think the state in this PR is okay:

flowchart TD
  ValidateDocument --> GetOperation
  subgraph ExecuteRequest
  GetOperation --> CoerceVariableValues
  CoerceVariableValues -->|is subscription?| CreateSourceEventStream
  end
  subgraph OperationExecution
  CoerceVariableValues -->|is query?| ExecuteQuery
  CoerceVariableValues -->|is mutation?| ExecuteMutation
  subgraph Subscribe
  CreateSourceEventStream --> MapSourceToResponseEvent
  end
  end

Loading

and it's significantly clearer than what we had before (which I can't draw a mermaid diagram of, so here's an awfully hacked together screenshot):