Resource registration#
As a Pulumi program is executed by the language host, it will make 📞 RegisterResource calls to the engine in order to declare resources and their desired state. Each 📨 RegisterResourceRequest contains:
The resource’s type and name.
The resource’s parent, if it has one.
A reference to the provider that manages the resource, if an explicit one has been specified. If no reference has been specified, Pulumi will use a default provider instance for the resource’s package and version.
Values for the resource’s input properties.
Any resource options that have been specified for the resource.
In order to determine what actions to take in order to arrive at the desired, the engine diffs the desired state of the resource against the current state as recorded in the state snapshot:
If there is no current state, the engine will attempt to create a new resource.
If there is a current state, the engine will diff the current state with the desired state to determine whether the resource is unchanged, requires updating, or must be replaced in some manner.
When the appropriate actions have been determined, the engine will invoke the relevant provider methods to carry them out. After the actions complete, the engine returns the new state of the resource to the program.
Although all of the above happens “in the engine”, in practice these concerns are separated into a number of subsystems: the resource monitor, the step generator, and the step executor.
Resource monitor#
The resource monitor (largely represented by resmon in the codebase; see
pulumi/pulumi:pkg/resource/deploy/source_eval.go) implements the
🔌 ResourceMonitor interface, which is the primary communication
channel between language hosts and the engine. There is a single resource
monitor per deployment. Aside from being a marshalling and unmarshalling layer
between the engine and its gRPC boundary, the resource monitor is also
responsible for resolving default providers and component
providers, responding to
📨 RegisterResourceRequests as follows:
The request is unmarshalled from the gRPC wire format into an engine-internal representation.
If the request lacks a
providerreference, the resource monitor will resolve a default provider for the resource’s package and version.If the request registers a remote component, the resource monitor will dispatch an appropriate 📞 Construct call to the component provider and await the result.
If the request does not register a remote component (a so-called custom resource, although this is in reality the default type of resource), the resource monitor will emit a
RegisterResourceEventand await a response.When a result is received (either in response to a đź“ž Construct call or a
RegisterResourceEvent), the resource monitor will marshal the result back into the gRPC wire format and return it to the language host.
Step generation#
The step generator (pulumi/pulumi:pkg/resource/deploy/step_generator.go)
converts RegisterResourceEvents and ReadResourceEvents from the Pulumi program
into executable steps. Steps are the internal representation of
everything that needs to happen to fulfill the goals of that program. The steps
are passed to the step executor, which will do the actual work encoded in the
steps.
This is a fire and forget process. Once a step has been generated
the step generator immediately moves on to the next RegisterResourceEvent. It
is the responsibility of the step executor to communicate the
results of each step back to the resource monitor.
executor to the step generator.
Process Overview#
Step Generation Algorithm#
When a RegisterResourceEvent is received:
URN Generation: Create URN from type, name, and parent
State Lookup: Find existing state by URN or aliases (error if multiple matches)
Import Check: If importing, generate
ImportStepand returnCheck Inputs: Call provider’s
Checkto validate and normalize inputsRun Analyzers: Execute configured analyzers for validation
Decision Point:
No existing state: Generate
CreateStepHas existing state: Proceed to diff
Diff: Compute differences between old and new state (can happen async, see below)
Generate Steps:
No changes:
SameStepUpdate:
UpdateStepReplace: Replacement flow (see below)
Note
Presently, step generation is a serial process (that is, steps are processed one at a time, in turn). This means that step generation is on the critical path for a deployment, so any significant blocking operations could slow down deployments considerably. In the case of an update, step generator latency is generally insignificant compared to the time spend performing provider operations (e.g. cloud updates), but in the case of a large preview operation, or an update where most resources are unchanged, the step generator could become a bottleneck.
There are some exceptions to this, e.g. with diffs (see below), but largely operations that happen in the step generator should be fast, to not slow the whole deployment down.
Read Resource Events#
While RegisterResourceEvents represent resources managed by Pulumi,
ReadResourceEvents fetch existing resources as external references. These
resources are marked External = true, meaning Pulumi tracks them but doesn’t
own their lifecycle.
Generate URN from type, name, and parent
Create state marked as
External = truewith provided IDCheck existing resource:
No existing: Simple
ReadStepto fetch stateExisting external: Simple
ReadStep(updating reference)Existing managed, same ID: “Relinquish” - release from Pulumi management, generate
ReadStepExisting managed, different ID: “Read replacement” - generate
ReadReplacementStepto delete old managed resource andReplaceStepmarker
Diffing#
While in most cases diffing boils down to calling a provider’s 📞 Diff method, there are a number of cases where this might not happen. The full algorithm that the engine currently implements is as follows:
If the resource has been marked for replacement out-of-band (e.g. by the use of the
--target-replacecommand-line option), the resource must be replaced.If the resource’s provider has changed, the resource must be replaced. Default providers are allowed to change without triggering a replacement if and only if the provider’s configuration allows the new default provider to continue to manage existing resources. This is intended to allow default providers to be upgraded without causing all resources they manage to be replaced.
If the engine is configured to use legacy (pre-1.0) diffs, the engine will compare old and new inputs itself (without consulting the provider). If these differ, the resource must be updated.
In all other cases, the engine will call the provider’s 📞 Diff method to determine whether the resource must be updated, replaced, or left as-is.
Parallel Diff Optimization#
To avoid blocking step generation, diffs can be computed in parallel using the
step executor’s worker pool (note that this is disabled by default as of the time
of this writing, and can be enabled with the PULUMI_PARALLEL_DIFF=true env var):
The DiffStep leverages parallel workers while the step generator continues
processing other resources. When complete, a ContinueResourceDiffEvent re-enters
step generation to produce the final steps.
Deletions#
After the program exits, resources not registered in the new state are deleted:
Replacement Decision Flow#
Pulumi supports two different type of replaces, either creating the new resource before deleting the old one (default), or deleting the old resource first, and then replacing it.
Delete-before-replace (DBR) is triggered when:
Provider indicates via
deleteBeforeReplacein 📨 DiffResponseUser specifies
deleteBeforeReplaceresource option
For DBR replacements, a second Check call is made without old inputs to ensure
new auto-generated properties (like names) don’t reuse old values.1Existing inputs may be used to repopulate default values for input
properties that are automatically generated when the resource is created but
that are not changed on subsequent updates (e.g. automatically generated
names). For replacements, we don’t want to reuse these values.
In such cases, it may be necessary to first delete resources that depend on that being replaced, since there will be a moment between the delete and create steps where no version of the resource exists (and thus dependent resources will have broken dependencies). The step generator does this as follows:
Compute the full set of resources that transitively depend on the resource being replaced.
Remove from this set any resources that would not themselves be replaced by changes to their dependencies. This is determined by substituting unknown values for any inputs that stem from a dependency and calling the provider’s 📞 Diff method.
Process the replacements in reverse topological order.
To better illustrate this, consider the following example (written in pseudo-TypeScript):
const a = new Resource("a", {})
const b = new Resource("b", {}, { dependsOn: a })
const c = new Resource("c", { input: a.output })
const d = new Resource("d", { input: b.output })
The dependency graph for this program is as follows:
We see that the transitive set of resources that depend on a is {b, c, d}.
In the event that a is subject to a delete-before-replace, then each of b,
c, and d must also be considered. Since b’s relationship is only due to
dependsOn, its
inputs will not be affected by the deletion of a, so it does not need to be
replaced. c’s inputs are affected by the deletion of a, so we must call
Diff to see whether it needs to be replaced or not. d’s dependency on a is
through b, which we have established does not need to be replaced, so d does
not need to be replaced either.
Dependent Replacement Algorithm#
Process:
Find all resources transitively depending on the DBR resource
Filter to only those with property dependencies (not just
dependsOn)For each dependent, test if it would be replaced:
Substitute unknowns for the DBR resource’s outputs
Call provider’s
Diffto determine if replacement needed
Sort remaining dependents in reverse topological order
Execute cascading replacements
Step execution#
The step executor is responsible for executing steps yielded by the step generator. Steps are processed in sequences called chains. While the steps within a chain must be processed serially, chains may be processed in parallel. The step executor uses a pool of workers to execute steps. Once a step completes, the executor communicates its results to the resource monitor. If a step fails, the executor notes its failure and cancels the deployment. Once the Pulumi program has exited and the step generator has issued all required deletions, the step executor waits for all outstanding steps to complete and then returns.
Examples#
The following subsections give some example sequence diagrams for the processes described in this document. Use your mouse to zoom in/out and move around as necessary.