---
title: ytt dev notes & tech forum discussions
tags: ytt
---

Note: This is a working doc meant for writing down working ideas. Please see the [docs](https://carvel.dev/ytt) for official definitions.

Input
---
Users:
- Configuration Authors: Authors of templates and data values. In Kubernetes world, these are often application developers, devops, or PAs. The authors then distribute the templates to those that want to install/use them.
- Configuration Consumers: Consumers of the templates that the Configuration Authors wrote. In k8s world, they use the templates to install the applications onto their clusters.

Types:
- NewBulkFilesSource: Used to process JSON input, from the ytt playground
- NewRegularFilesSource: Primary input source. Processes files input with `-f` and accepts other flags.

- Directory Structure / file system contract:
- Libraries:
- Like a virtual file system built from the relative file path of the inputs. Some libraries can be 'private'.
- Private library:
- is one that is in the `_ytt_lib` directory.
- Other libraries cannot access a private library.
- It can be thought of as a separate ytt invocation.

- File Marks: Used to override the default behavior of ytt file processing. Can override the file extension.
- Data Values: Provide a way to inject input data into a template via parameterized variables.

Concepts
---
- Overlays:
- Additional templates that are merged with the previous templates. Templates and data values are merged separately.
- Left: being modified
- Right: the overlay describing the modification.

- Single unit template: Each yaml document is its own single unit. One unit can overlay another, and does not relate to scope. For other file types, a file may be a single unit of configuration.

Strive for:
---
- Readability:
- Important for new maintainers
- Descriptive & consistent variable names
- Extract code to functions when it's not helpful to have the context at the current level.
- Don't extract code to function when it does not add to complexity.
- Refactors are necessary to get here but should be done separate from features.

- "Single Point Of Truth": Goal is to not have duplicate code to reduce inconsistency and organization. Don't pay the 'complexity cost' twice.

- Compactness: the property that a design can fit inside a human being's head.

- Be different when we have a reason to be.

- Order test variables consistently across tests.

- If I remove one line/thing in the code only ONE test should fail. In other words, test only one unit per test.

:bee: :bee: :bee:
Go patterns
---
- Explicit error checks
- Discourage use of named returns generally
- generally, yes, adds work to the reader to determine what's being returned.
- when small, might be nice to use to trim
- no reason can't "document" return values with named returns (don't have to use them)
- Keep methods private by default
- this makes sense in general, however it may be worth considering what the package 'api' should expose. Could the package be imported by other tools?
- Function names plurality match what they return (ie: return list, name is plural)
- Constructors start with 'New'

(not yet discussed) ytt patterns
---
- library patterns
- A file (schema or data value) uses it's `libRef` to indicate what library it belongs to. No libRef? Then it's intented for the current library.
- A private library is mostly 'self contained' and can be thought of as a separate ytt invocation.
- Many of data value and schema code follows a similar pattern.. Is this the goal or an intermediate stage?
- Factory
- LibraryExecutionFactory
- Envelope
- DocumentSchemaEnvelope
- Parameterized tests
- yamltemplate package: tests that can be in a single file. Similar to e2e tests, asserts on output. (used for overlays, ytt starlark addtions, file positions via `--output`)
- TestSchema_With_fuzzed_inputs ?

AST Structures
---
What is it?
* Composite of the node
* tree structure
* A scalar isn't a Node, it's the value of a Node

How're they created?
* how do they go from bytes to ast?
* Scalar types are parsed using our forked yamlv2 parser
* workspace package templateloader.go -> ParseYAML()
* yamlmeta package document_set.go -> NewDocumentSetFromBytes() unstructured data becomes structured data
* parser.go -> ParseBytes() -> parseBytes()
* Decode() yields raw yaml structures (no annotations yet)
* yaml.v2 package yaml.go unmarshals the data
* yaml.v2 defines a struct that represents the structure of the type and the decoder parses based on that
* out is what we write the value to
* n is the data we're decoding
* yamlmeta package parser.go -> parser()
* output is either a map, with map items or array with array items
* we use a mapSlice instead of a map to preserve map ordering
* assignMetas() takes yaml comments and converts to metas
* comments directly to the left of a node OR comments directly after a node are added to the node as meta
* We check that a map items' value is null if it is followed by #@ value
* Since yaml.v2 does not set the position of maps, we manually set the position of a map to the position of its parent

Do they contain scalars?
* Yes, but not as a separate node.
* They are stored in the value of where they are contained, (ie. Document, MapItem, and ArrayItem)this is where they get their position as well.
* Scalars implement the TypeWithValues interface, which has a GetPosition() function.

What do they store?
* Positions
* GetValues returns the children
* metas

How are they output?

### Representation of a computed node value:

---
foo:
- { a: null } #@ 41+1

doc
 -> map
    -> mapitem("foo")
       -> array
          [0] map
              -> mapitem("bar")
                   value: null
                   meta: "@ 41+1"