<style> .ui-infobar, #doc.markdown-body { max-width: 1000px; } </style> # Node Socket Type System Inferrence Node socket types are mostly static, but some aspects of it are dynamically inferred. ## Field Type Inference _Field Types_ in geometry nodes are the main example right now: Whether a socket represents a single value, a field, or either of these (undecided) is determined during node tree updates ([update_field_inferencing](https://projects.blender.org/blender/blender/src/commit/f2aab0776047396c99c781eb0ad0a6d1ffadcd3d/source/blender/blenkernel/intern/node_tree_field_inferencing.cc#L699)). This information is used to show invalid field connections (field -> single value), which are marked as invalid links[^1]. In the case of field evaluation the execution code itself handles invalid connections without relying on the preprocessing step: if a single-value node input is connected to a field source it will simply receive the default value of zero when reading that input. This may not be desirable if a value of zero could have unintended consequences. The execution system may need to be more strict about violated type constraints and disable parts of the execution accordingly. ## Data Sockets Field sockets create virtual arrays based on the context in which they are evaluated. An field input will either be available in the context and output an array of the expected size, or simply return a default value if they can't be used in the context. Volume grid sockets work differently from regular fields. Topology is not determined by context: how big the grid is, which voxels are active, etc. is based on the connected input grids. The default behavior is that combining multiple grids will create a _topology union_ of all input grids. Combining grid data and fields has to follow conversion rules: Grids are a _narrower_ type than fields, meaning that they cannot simply be interpreted as fields. Fields, on the other hand, can be evaluated in a grid context, using the set of active voxels as the domain size. ![Screenshot_20231114_113314](https://hackmd.io/_uploads/SJpxp6x4T.png) Future data sockets, such as lists, would also be a more specialized type that cannot be interpreted as a field. Generalizing the type inference mechanism should make the addition of such socket types easier. ## Type Constraints The rules for field type inference are expressed as dependencies between node input and output sockets. Together with the links between nodes these dependencies define which field types are admissible. Definition of socket dependencies for inputs and outputs[^2]: ```cpp enum class InputSocketFieldType { /** The input is required to be a single value. */ None, /** The input can be a field. */ IsSupported, /** The input can be a field and is a field implicitly if nothing is connected. */ Implicit, }; enum class OutputSocketFieldType { /** The output is always a single value. */ None, /** The output is always a field, independent of the inputs. */ FieldSource, /** If any input is a field, this output will be a field as well. */ DependentField, /** If any of a subset of inputs is a field, this out will be a field as well. * The subset is defined by the vector of indices. */ PartiallyDependent, }; ``` Connections between nodes represent simple assignments, so the field type on the input side has to be compatible with the field on the output side. ## Inference Algorithm The field type inference algorithm resolves field types of sockets to their _widest_ possible type, i.e. sockets become fields if they have to, but otherwise use single values. The algorithm is a _constraint propagation_ method, utilizing connections between nodes to reduce complexity: after topological ordering all constraints can be solved in a limited number of stages. These stages may need to be repeated a number of times because of cyclic dependencies that can be introduced by simulation/repeat zones. The number of passes required is at most the number of nested zones in a tree. 1. Right-to-Left Data Requirements 2. Group Input Constraints 3. Left-to-Right Field Status Propagation 4. Group Output Constraints Each stage can override the field state of sockets from the previous stage. This ranks different constraints by handling them in order of increasing importance (later constraints override earlier ones). Any remaining conflicts at the end are considered errors (invalid links) and have to be resolved by the user. ### Proposal 1: Simplify Field State Field _state_ during the propagation is stored with 4 boolean flags ([SocketFieldState](https://projects.blender.org/blender/blender/src/commit/f27dd74986b6157f949d8599fd4d3c9831d83934/source/blender/blenkernel/intern/node_tree_field_inferencing.cc#L147-L158)). However, most possible flag combinations are never used, which suggests a simpler representation is possible. - `is_field_source`, _"This socket starts a new field."_ This flag never gets propagated to other sockets. Set only for 1. outputs with the `FieldSource` field type. 2. group inputs supporting fields. By assuming the narrowest type some errors in groups can be detected without knowing the full context. - `is_always_single`, _"This socket can never become a field, because the node itself does not support it."_ This flag never gets propagated to other sockets. Set only for inputs and outputs with the `None` field type. - `is_single`, _"This socket is currently a single value. It could become a field though."_ The actual field state, if the socket is not a single value it becomes a field. True by default, single values are the most compatible type. False for group inputs that support fields, those are assumed to be fields for best compatibility. Forced to true if `is_always_single` is true. Propagated left-to-right unless target inputs are fields or internal input dependencies are fields. - `requires_single`, _"This socket is required to be a single value. This can be because the node itself only supports this socket to be a single value, or because a node afterwards requires this to be a single value."_ Propagated right-to-left from inputs to outputs if any connected input is true. Implicit field inputs force connected outputs to `false`, while single value outputs force connected inputs to `true`. :warning: Naming could be better: Can also be described as "can use a single value": If true then a single value field type is possible. The socket does not "require" a single value so much as "allowing" it, and we chose single values by default since they are the widest field type. :warning: This value never changes based on `is_single`, so it could be calculated separately in advance of the actual field type propagation. In summary: There are only two mutable values relevant to propagation state: - `requires_single`: Should be called `allows_single`, and calculated in a separate pre-processing step. - `is_single`: The current state of the field type resolution. The modified procedure would look like this: 1. Initialize fixed field types based on single-value sockets and field sources (`is_always_single`, `is_field_source`). 2. Propagate constraints to determine which sockets accept single-value (`requires_single`). Only requires right-to-left propagation. 3. Resolve actual field state (`is_single`). Only requires left-to-right propagation. [^1]: Links with invalid field connections are not actually marked "invalid" with the `NODE_LINK_VALID` flag, but rather are only drawn red based on socket symbols: [link 1](https://projects.blender.org/blender/blender/src/commit/f2aab0776047396c99c781eb0ad0a6d1ffadcd3d/source/blender/editors/space_node/drawnode.cc#L2302-L2309), [link 2](https://projects.blender.org/blender/blender/src/commit/f2aab0776047396c99c781eb0ad0a6d1ffadcd3d/source/blender/editors/space_node/drawnode.cc#L2102-L2111), [link 3](https://projects.blender.org/blender/blender/src/commit/f2aab0776047396c99c781eb0ad0a6d1ffadcd3d/source/blender/blenkernel/intern/node_tree_anonymous_attributes.cc#L25-L28). [^2]: These are not actually field "types" per se but field _dependencies_. ### Constraints Node groups should behave the same way a built-in node would. Group input/output constraints have highest priority for this reason. |Constraint|Propagation|Priority| |-|-|-| |Inputs are single-value if the declared type is `None`|Local|1| |Inputs are fields if the declared type is `Implicit` and they are _unconnected_|Local|1| |Outputs are single-value if the declared type is `None`|Local|1| |Outputs are fields if the declared type is `Source`|Local|1| |||| |Group inputs that support fields have to be fields (widest possible input)|Local|$\infty$| |Group inputs that don't support fields have to be single-value [^3]|Local|$\infty$| |||| |Outputs connected to single-value inputs have to be single-value|Right-to-Left|2| |Outputs depending on any field inputs have to be fields|Left-to-Right [^4]|3| |Inputs connected to a field output have to be fields|Left-to-Right|2| |Inputs with a dependant single-value output have to be single-value|Right-to-Left|3| [^3]: This may be affected by changes from R2L propagation: a group input can be limited to single-value by one connection first and then propagate that L2R via another connection. [^4]: This constraint is handled both in R2L and L2R propagation. Probably to constraint ordering simple, otherwise would have to have an L2R pass for implicit field propagation first. Feels like that should be a simple input->output field propagation and the implicitness of the input should not matter. ### Possible socket states Current: |`always_single`|`field_source`|`requires_single`|`is_single`|Possible State?|Comment| |-|-|-|-|-|-| |0|0|0|0|<span style="color:green">yes</span>|| |0|0|0|1|<span style="color:red">no</span>|`is_single` but neither `always_single` nor `requires_single`| |0|0|1|0|<span style="color:red">no</span>|`requires_single` can't be ignored| |0|0|1|1|<span style="color:green">yes</span>|| |0|1|0|0|<span style="color:orange">yes</span>|trivial: `field_source`| |0|1|0|1|<span style="color:red">no</span>|`field_source` and `is_single` conflict| |0|1|1|0|<span style="color:orange">yes</span>|meaningless: `requires_single` should be ignored| |0|1|1|1|<span style="color:red">no</span>|`field_source` and `is_single` conflict| |1|0|0|0|<span style="color:red">no</span>|`always_single` but not `is_single`| |1|0|0|1|<span style="color:orange">yes</span>|trivial: `always_single`| |1|0|1|0|<span style="color:red">no</span>|`always_single` but not `is_single`| |1|0|1|1|<span style="color:orange">yes</span>|trivial: `always_single`| |1|1|0|0|<span style="color:red">no</span>|`field_source` and `always_single` conflict| |1|1|0|1|<span style="color:red">no</span>|`field_source` and `always_single` conflict| |1|1|1|0|<span style="color:red">no</span>|`field_source` and `always_single` conflict| |1|1|1|1|<span style="color:red">no</span>|`field_source` and `always_single` conflict|
Last changed by  
 
lukas-tonne
146

Read more

Investigation: Cage Deformation with Grids

Problem Statement: Samples inside a mesh should follow the deformation of the mesh volume.

11/30/2023
Volume Attributes MVP

Make volumes in geometry nodes more useful and flexible.

10/6/2023
High level controls for node assets

A proposal for making complex node trees customizable using node groups parameters. Node assets that are supposed to be used as building blocks require detail knowledge to tweak their internals. Some parameters can be exposed on the modifier level, but extending a system requires a different approach, as will be demonstrated below. This proposal describes a mechanism to allow users to specify node groups at the highest level of a node asset, which are then used inside the asset without requiring users to touch the asset internals themselves. The modifier stack for node assets and its limitations Simon Thommes recently published the first set of hair hair assets to be shipped with Blender. Simon breaks down the hair workflow into separate modifiers. This relies a lot on the fact that modifiers can be applied as a single self-contained step one after the other.

5/12/2023
Function Socket Notes

Function sockets are single-valued (no fields). This should keep overhead from type checking and dispatching to a minimum. That does not mean function sockets cannot take and return fields, it just means the function itself is the same for all elements of a field. Data flow for functions in geometry nodes flowchart G[Node Tree]:::data F[Default Bindings]:::data G --> E([bind]):::op F --> E E --> D[Closure]:::data D -->|passed through the node tree| B([evaluate]):::op

5/12/2023
Read more from lukas-tonne
Published on HackMD