Xlator options update guide

This is a guide to help update xlator options to include new fields required for GD2.

The aim for migration is to make information currently presently available in the volume options map in GD1, into the xlators themselves. GD2 will load this information directly from the xlators.

In addition, we will also attempt to identify a few special options that require additional help from GD2.

Updating Options

We will be working on the experimental branch for now, and will cherry pick the changes onto master later. Ensure you branch off from experimental before continuing.

• Choose the xlator you want to work on. Mark yourself against the xlator in the xlator list below.
• Open the glusterd volume options table (glusterd_volopt_map). This is available in the file xlators/mgmt/glusterd/src/glusterd-volume-set.c.
• Open the file with the xlator options table (usually <xlator>.c)
• Search the glusterd map, for options of the xlator you chose.
• For each option you find in the map,
  • If the option has been flagged 'VOLOPT_FLAG_XLATOR_OPT' ignore the option
  • Find the relevant option in the xlator options table, by searching for the same key
    • Sometimes, you might need to search using the value of the .option field.
  • Update the new fields for the option in the xlator options table. Information about the new fields is available as comments on the new fields.
    • .op_version - A single entry array, with the value copied from the map
    • .flags - Bitwise ORed opt_flags_t. Always set OPT_FLAG_SETTABLE. OR any existing flags from the map. If the map has .type set with *_DOC or GLOBAL_*, add the DOC and GLOBAL flags.
    • .tags - An array of descriptive tags that can be used to group and filter options in documentation. A list of existing tags can be found here. If you add additional tags, add them to the list linked above.
    • .validate_fn - Copy this over now, but keep the field commented out.
  • Ensure that your option has a default value set.

After all the options found in the glusterd map have been updated, there might options left in xlator options table that haven't been updated. For each of these options,

• If the option is suitable for being user settable, update the option as described above. Set op_version to GD_OP_VERSION_4_0_0
• If it shouldn't be user settable, ensure a valid default value is set.
• If a default value can't be set and it is required that GD2 set them, follow instructions for variable strings.

Once all the changes have been made, commit and add 'Updates #302' in the commit message to link to the issue being used to track this effort. Send the patch for review and add the review link to the list

If there are any questions or comments, add them in the section below.

List of xlators

Xlator	Worked On By	Patch	Comments
cluster/afr	Ravi	https://review.gluster.org/#/c/18172/
cluster/dht
cluster/ec
cluster/stripe	Amar		Done
debug/error-gen
debug/io-stats
debug/sink
debug/delay-gen	Pranith	https://review.gluster.org/19122
debug/trace
encryption/crypt
encryption/rot-13
features/arbiter	Ravi		Does not have any volume options.
features/barrier	Atin	https://review.gluster.org/#/c/18158/
features/bit-rot
features/changelog	Kaushal	https://review.gluster.org/18472
features/changetimerecorder
features/compress
features/gfid-access
features/glupy
features/index	Kaushal	https://review.gluster.org/18467
features/leases
features/locks
features/marker	Sanoj	https://review.gluster.org/#/c/18170/
features/quiesce	Amar		Done
features/quota	Sanoj	https://review.gluster.org/#/c/18170/
features/read-only	Karthik	https://review.gluster.org/#/c/18166/
features/shard
features/snapview-client	Sunny
features/snapview-server	Sunny
features/trash	jiffin	https://review.gluster.org/18160
features/upcall	skoduri	https://review.gluster.org/#/c/18159
features/selinux	jiffin	https://review.gluster.org/18160
features/sdfs
meta	Rafi KC	–––––––––––––––––––––-	No updation required
mgmt/glusterd			Not needed
mount/fuse
nfs/server			Possibly not needed
performance/decompounder
performance/io-cache	Rafi KC
performance/io-threads	Pranith	https://review.gluster.org/19120	TODO: Do we need anything specific for it to be working fine on both client and server side
performance/md-cache	Poornima	https://review.gluster.org/18162
performance/nl-cache
performance/open-behind	Milind	https://review.gluster.org/18408
performance/quick-read	Poornima	https://review.gluster.org/18164
performance/read-ahead
performance/readdir-ahead
performance/symlink-cache
performance/write-behind
playground/template
protocol/auth	Milind	https://review.gluster.org/19182
protocol/client	Kaushal	https://review.gluster.org/18465
protocol/server	Kaushal	https://review.gluster.org/18466
storage/bd
storage/posix	Kaushal	https://review.gluster.org/18461
system/posix-acl	Raghavendra Talur
experimental/fdl	Rafi KC
experimental/jbr-client	Rafi KC
experimental/jbr-server	Rafi KC
experimental/posix2
experimental/rio
rpc	Milind

List of tags

A list of known tags that can be used to tag options. If you tag your option with a new tag, add it to this list.

• perf
• cache
• cacheconsistency
• cachetimeout
• dev
• debug
• linux
• nfs
• samba
• container
• security
• backup
• upcall
• replicate
• halo
• journal
• georep
• glusterfind

Variable Strings

There are some options that are not user settable, but still need to be set by GD2 for the xlator to work. These options also cannot have a default value set, as the value depends on the volume information.

GD2 will define a set of replaceable strings. Xlators can use these strings in the default values of their options, and GD2 will replace the string with proper information when generating the volfiles.

For eg., the storage/posix xlator has the option directory. This options isn't user settable, changes for each volume(/brick) and needs to be set for the xlator to work. The default value for this can be set to the variable string "{{brickpath}}", to let GD2 know that it should be replaced with the relevant value.

As GD2 doesn't have a list of these options or variable strings yet, we are building the list.

For any option of this type you find,

• Search for an applicable variable string from the list, and set it as the default value for the option
• If a relevant string isn't found, define one yourself.
• Add the option and the variable string to the list

List of variable strings

Xlator	Option	Expected value	Expected VarString	Settable	Comments
posix	brick-path	Brick path	{{brickpath}}
changelog	changelog-brick	Brick path	{{brickpath}}
changelog	changelog-dir		{{brickpath}}/.glusterfs/changelogs	Yes
bitrot-stub	export	Brick path	{{brickpath}}	No
ctr	db-path	?
ctr	db-name	?
ctr	hot-brick	on			will be set by glusterd because of another xlator
quota		server-quota	??(needs to be cleaned .. stale)
quota		volume-uuid	uuid of brick
marker	timestamp-file	??
marker	volume-uuid	uuid of brick
marker	quota-version	needs to be read from quota conf file
protocol/server	conf-dir	config-directory
protocol/server	trace
protocol/server	verify-volfile-checksum
protocol/server	volume-filename
disperse	redundancy	This should be set based on the cli option
disperse	iam-self-heal-daemon	?

Questions / Comments

• How would you handle conflicting options?
• How would you handle closely related options, options that should be set and unset together
• How to target individual xlators occuring in both client and server graphs? eg. io-threads
• In case option has multiple keys, GD2 should set first option using first key only. Xlators should always lookup options using the first key (in GF_OPTION_INIT/RECONF).
• The validations done for each xlator in glusterd_op_stage_set_volume() will ported to glusterd2 as well?