---
title: Emitting comments from ytt templates
tags: ytt
---

## Summary

`ytt` should have the facility to safely emit YAML comments from YAML templates.

Today, the people who need to read and understand the _output_ of ytt templates are having a bad time.
Without the ability to weave in context for the humans (in the form of YAML comments), **they must
burn hours sleuthing** back through whatever process(es) executed ytt, to find their inputs.

Better would be the ability to include this additional information in the form of comments in the output.
They could not only leave more explicit breadcrumbs, but name the source of a change or rationale for a
given value. This not **reduces the guesswork**, but **eliminates much of the toil** in gathering the facts.

...

Also, folks who want to automate building ytt templates also got it rough. Without the ability to include
ytt templating directives _as output_ of a ytt template, they are **forced to hack** a makeshift templating
system together. At best it makes the whole system **avoidably complex** and **unnecessarily brittle** as they
must bring in other tools (e.g. sed/awk/yq/patch).

Better would be the ability to emit ytt annotations and code in the form of comments in the output.
This would allow folks to design ytt templates that evaluate in phases. Each phase is a ytt invocation
with data values specific to that phase. All along the way, they **continue to reap the benefits** of
structural templating. The resulting system is **conceptually simpler**: one tool.

_(TODO: properly acknowledge the dangers of generating templating.)_

Prototype work-in-progress: https://github.com/vmware-tanzu/carvel-ytt/pull/638

## Proposal

### Emitting YAML Comments

Authors can template in comments via the `@yaml/yaml-comment` annotation.

databases:
  #@yaml/yaml-comment "selected '{}' because env = '{}'".format(data.values.db_conn[data.values.env].name, data.values.env)
  name:
    alias: #@ data.values.db_conn[data.values.env].name
    hostname: #@ data.values.db_conn[data.values.env].host

By default, the comment is prefixed with a leading space:

databases:
  # selected 'dev1' because env = 'dev'
  name:
    alias: dev1
    hostname: myexampledb.a1b2c3d4wxyz.us-west-2.rds.amazonaws.com

Authors may opt-out of emitting this leading space in corner-case

It is disallowed to emit `ytt` templating through the `@yaml/yaml-comment`:

databases:
  #@yaml/yaml-comment "@overlay"
  name:
    alias: #@ data.values.db_conn[data.values.env].name
    hostname: #@ data.values.db_conn[data.values.env].host

### Emitting ytt templating

- using sed magic or
- giving up on ytt and using jinja, yq

Use cases:
1. generate ytt with ytt:
- in the package world — automating the process of creating packages.
- is doing package installs in packages so that his customers install just one package.
2. education of the team/customers of using ytt. Having comments in output inline is much easier
- demo magic — create scripts inside it and some functions and it has your whole demo scripted.
- With attribution (e.g. from a particular overlay).
- GitOps approach to k8s.
- ytt output to a git repo.
- this repo is documentation for people who are operating the platform. This saves the toil for reconstructing from where specific/custom values come from.
- overlays adding comments?

Other Use-Cases?:
- preserve input comments
- `yaml.decode()` / `yaml.encode()` to be able to apply overlays on YAML that contains ytt annotations?

#@overlay/match ...
---
#@yaml/ytt-annotation "overlay/match by= "
#@yaml/ytt-code       "if "
#@yaml/comment        "because env = "+ data.values.env + "; disabling bar." 
foo:
  #@yaml/comment "null set by not-bar.yaml"
  bar: #@yaml/ytt-value ""

#@overlay/match ...
---
#@yaml/ytt-comment "! replace the business with the other thing."
#@yaml/ytt-comment "@overlay/match ..."
#@yaml/ytt-comment "@ if "
#@yaml/comment "because env = "+ data.values.env + "; disabling bar." 
foo:
  #@yaml/comment "null set by not-bar.yaml"
  bar: #@yaml/ytt-value ""

#@overlay/match ...
---
#@yaml/comment "! replace the business with the other thing."
#@yaml/comment "@overlay/match ..."
#@yaml/comment "@ if data.values.env == \"prod\":"
#@yaml/comment " because env = "+ data.values.env + "; disabling bar." 
foo:
  #@yaml/comment "null set by not-bar.yaml"
  bar: #@yaml/ytt-value ""

#@overlay/match ...
---
#@yaml/comment "replace the business with the other thing.", no_leading_space=True
#@yaml/comment "@vrabbi: replace the business with the other thing.", no_leading_space=True      # error: can't write ytt comments; use ytt/comment to do so.
#@yaml/comment "@vrabbi: replace the business with the other thing."
#@yaml/comment "@ytt:library"
#@ytt/comment "@overlay/match ..."
#@ytt/comment "@ if data.values.env == \"prod\":"
#@ytt/comment "@ if data.values.env == \"prod\":"
#@ytt/comment "! because env = "+ data.values.env + "; disabling bar." 
#@ytt/comment " because env = "+ data.values.env + "; disabling bar."   # error: unknown comment; start with # or !; for YAML comments use yaml/comment.
foo:
  #@yaml/comment "null set by not-bar.yaml"
  bar: #@yaml/ytt-value ""

#@yaml/yaml-comment "replace the business with the other thing.", no_leading_space=True
#@yaml/yaml-comment "@vrabbi: replace the business with the other thing.", no_leading_space=True      # error: can't write ytt comments; use yaml/ytt-comment to do so.
#@yaml/yaml-comment "@vrabbi: replace the business with the other thing."
#@yaml/yaml-comment "@ytt:library"

#@yaml/ytt-comment "@overlay/match ..."
#@yaml/ytt-comment "@ if data.values.env == \"prod\":"
#@yaml/ytt-comment "@ if data.values.env == \"prod\":"
#@yaml/ytt-comment "! because env = "+ data.values.env + "; disabling bar." 
#@yaml/ytt-comment " because env = "+ data.values.env + "; disabling bar."   # error: unknown comment; start with # or !; for YAML comments use yaml/plain-comment.
foo:
  #@yaml/comment "null set by not-bar.yaml"
  bar: #@yaml/ytt-value ""

#@yaml/comment " replace the business with the other thing."
#@yaml/ytt-comment "@overlay/match ..."
#@yaml/ytt-comment "@ if data.values.env == \"prod\":"
#@yaml/ytt-comment "! because env = "+ data.values.env + "; disabling bar." 
foo:
  #@yaml/comment "null set by not-bar.yaml"
  bar: #@yaml/ytt-value ""

#cloud-config:

# replace the business with the other thing.
#@overlay/match ...
#@ if data.values.env == "prod":
#! because env = dev; disabling bar. 
foo:

#@overlay/match ...
---
#@yaml/ytt-comment "! replace the business with the other thing."
#@yaml/ytt-comment "@overlay/match ..."
#@yaml/ytt-comment "@ if "
#@yaml/comment "because env = "+ data.values.env + "; disabling bar." 
foo:
  #@yaml/comment "null set by not-bar.yaml"
  bar: #@yaml/ytt-value ""

---
#@overlay/match
#@ if
#because env = dev; disabling bar.
foo:
  # null set by not-bar.yaml
  bar: null

def generate_my_template():
return """
#@ load("@ytt:data", "data")
#@overlay/match
---

"""
end

#@ load("mytemplate.star", "generate_my_template()")

#@yaml/comment generate_my_template()

## Implementation considerations

- for corner cases, one can use text templating
- what about multi-line strings?
- how does this affect the `ytt fmt` command?

## Unrelated concerns

Issues with Tanzu TKG extensions:
- LDAP configuration in Grafana confg

I have applied and succeeded in <a href="https://quikdraw.io">Quick Draw</a>