# Release notes for Gluster 4.0.0

** DRAFT DRAFT ! DRAFT !!!**

**Header for the release - TBD**

The most notable features and changes are documented on this page. A full list
of bugs that have been addressed is included further below.

- [Major changes and features](#major-changes-and-features)
- [Major issues](#major-issues)
- [Bugs addressed in the release](#bugs-addressed)

Further, as 3.13 is a short term maintenance release, features included
in that release are available with 4.0.0 as well, and could be of interest to
users upgrading to 4.0.0 from older than 3.13 releases. The 3.13 [release notes](http://docs.gluster.org/en/latest/release-notes/)
captures the list of features that were introduced with 3.13.

**NOTE:** As 3.13 was a short term maintenance release, it will reach end of
life (EOL) with the release of 4.0.0. ([reference](https://www.gluster.org/release-schedule/))

## Major changes and features

Features are categorized into the following sections,

- [Management](#management)
- [Monitoring](#monitoring)
- [Performance](#performance)
- [Geo-replication](#geo-replication)
- [Standalone](#standalone)
- [Developer related](#developer-related)

### Management

#### 1. GlusterD2
**Notes for users:**
- Need GD2 team to fill in enough links and information here covering,
- What it is
- Install and configuration steps
- Future plans (what happens in 4.1.0 and further)

**Limitations:**

**Known Issues:**

### Monitoring

The lack of live monitoring support on top of GlusterFS till date was a limiting factor for many users (and in many cases for developers too). Statedump did some work of helping during debugging, but was bit heavy for monitoring live.

The existence of `debug/io-stats` translator was not known to many and `gluster volume profile` was not recommended as it took a hit on performance. With this release, glusterfs's core infrastructure itself gets some mechanisms to provide internal information.

There are few Github Issues which talk about the feature in detail:

#### 1. More metrics on fops at every translate layer
- https://github.com/gluster/glusterfs/issues/137
- Now, frame latency at every translators in stack can be measured, and is enabled by default. This helps to understand the usage pattern and also to debug any performance issues faster.

**Limitations:**
- Gluster Engineering doesn't recommend disabling the feature, and in 4.0 version there is no way to turn this off.
- By 4.1 version, there would be a CLI option to disable the feature (for the advanced users)

#### 2. Monitoring support
- https://github.com/gluster/glusterfs/issues/168
- Currently, only project which consumes the metrics and provides basic monitoring is https://github.com/amarts/glustermetrics, which provides a good idea on how to utilize the metrics dumped from the process.
- Users can send SIGUSR2 signal to the process to dump the metrics, in `/var/run/gluster/metrics/` directory.
- NOTE: this is actually, 'WORKDIR/metrics'

**Limitations:**
- Currently only Gluster's Stack and Memory management systems are dumping the metrics
- For other translators and other core components, framework to provide more metrics already exists, but code is not yet complete.

**Known Issues:**
- As discussed in Mailing list ([here](http://lists.gluster.org/pipermail/gluster-devel/2017-August/053426.html) and [here](http://lists.gluster.org/pipermail/gluster-devel/2017-September/053593.html)), there are better ways than sending signal to dump metrics. But that is not yet complete. Sending signal was choosen as an easier/simpler way to demonstrate the feature.

<Do we need any screenshots of how the latency graph looks like?

### Performance
#### 1. EC: Make metadata [F]STAT/[F]GETXATTR operations faster
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/368
- Auto enabled, no options or documentation required
- Release notes:
- Can be handled by the release team

**Limitations:**

**Known Issues:**

#### 2. To make md-cache capable to serve nameless lookup
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/232
- Release notes:
- Need to have some detail on what workloads will stand to gain by this change

**Limitations:**

**Known Issues:**

#### 3. md-cache: Allow runtime addition of xattrs to the list of xattrs that md-cache caches
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/297
- Needs user documentation for option performance.xattr-cache-list
- Release notes:
- Need documentation of the option, and also some details of when to use this feature
- Maybe better if this appeared in the documentation and we could point to the same

**Limitations:**

**Known Issues:**

#### 4. Cache last stripe of an EC volume while write is going on
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/256
- Release notes:
- Needs option documentation and also use-case details, as to when to enable this

**Limitations:**

**Known Issues:**

#### 5. tie-breaker logic for blocking inodelks/entrylk in SHD
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/354
- Release notes:
- Internal feature to enable faster(?) self-heal, mention it as such

**Limitations:**

**Known Issues:**

### Geo-replication
#### 1. JSON output for Geo-rep status and config for Glusterd2 integration
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/361
- Added option --json to the gsyncd.py script
- Are end users going to use the status command here?
- Documentation update?
- Release notes:
- If there is no end user impact, no specific notes are needed for the same

**Limitations:**

**Known Issues:**

#### 2. Enhance Geo-replication to use Volinfo from Config file
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/396
- Documentation needs to be updated with the changed conf file section
- Release notes:
- Need documentation of explanation of the process in the notes, and also when to use the same

**Limitations:**

**Known Issues:**

#### 3. Geo-replication: Improve gverify.sh logs
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/395
- Release notes:
- Needs documentation around new log file locations and possibly strings to check for mount failures etc.

**Limitations:**

**Known Issues:**

#### 4. Geo-rep: Cleanup stale (unusable) XSYNC changelogs.
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/376
- Release notes:
- A note on change in behavior maybe needed

**Limitations:**

**Known Issues:**

#### 5. Improve gsyncd configuration and arguments handling
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/73
- Release notes:
- Needs user facing documentation for newer options and such
- There seems to be code improvement as well in the patches, so that may not be needed in the release notes

**Limitations:**

**Known Issues:**

### Standalone
#### 1. Ability to force permissions while creating files/directories on a volume
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/301
- Adds options,
- "create-mask" and "create-directory-mask"
- "force-create-mode" and "force-create-directory"
- End user documentation is required
- Release notes:
- Need a summary of what the option would enable and how to set the same
- If relevant user documentation is added, we maybe able to point to the same as well

**Limitations:**

**Known Issues:**

#### 2. Replace MD5 usage to enable FIPS support
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/230
- Need a lot of context on options and versions before which this feature can be used
- Release notes:
- Need help from authors to complete

**Limitations:**

**Known Issues:**

#### 3. Dentry fop serializer xlator on brick stack
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/397
- Release notes:
- Option can be turned on by `gluster volume set $VOLNAME features.sdfs enable`
- This feature makes sense when there are lot of renames and directory/file creates and delete operations on the mountpoints.

**Limitations:**
- Considering the feature takes lock for given directory operations, there may be a minor hit in performance compared to disabled config.
- It is still recommended to enable this feature in those use-cases than looking for performance, as it helps resolve many issues which otherwise can't be solved.

**Known Issues:**
- Performance hit with this patch is not yet measured.

#### 4. Add option to disable nftw() based deletes when purging the landfill directory
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/371
- Release notes:
- Note that the option to enable this feature is called `disable-landfill-purge`, and should be enabled in `storage/posix` translator in volume file.
- By default the value of this is `off`.

**Limitations:**
- With the current version of glusterd and GD2, this feature can't be enabled using CLI.
- If this option is enabled, then there will be more space consumed in backend as self-heal wouldn't delete any directories.

**Known Issues:**
- Currently to enable this feature, one should make code change, and re-compile.

#### 5. Add option in POSIX to limit hardlinks per inode
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/370
- Release notes:
- Use github description

**Limitations:**

**Known Issues:**

#### 6. Enhancements for directory listing in readdirp
**Notes for users:**
- https://github.com/gluster/glusterfs/issues/239
- Potentially has some behavioral change in the way rebalance works, needs some documentation efforts there
- Release notes:
- Note the changes in rebalance behavior, any performance gains?

**Limitations:**

**Known Issues:**

### Developer related
#### 1. xlators should not provide init(), fini() and others directly, but have class_methods
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/164
- Before this release, there was a hard requirement on few of the methods being present in an xlator even if they are doing nothing.
- Also, when a translator's .so file needed to be `dlopen()`, many symbols were read from it.
- With the patch https://review.gluster.org/18876, expectation is one just needs to export 'xlator_api' structure with all the init()/fini() etc methods initialized.
- Release notes:
- There is no hard stop on supporting old symbols yet, but advise is to move to new symbols.

#### 2. Framework for distributed testing
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/374
- Developer README already present
- Release notes:
- Can be handled by the release team

#### 3. New API for acquiring mandatory locks
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/393
- Code comment for API exists, and release notes can point to that
- Release notes:
- Can be handled by the release team

#### 4. New on-wire protocol (XDR) needed to support iattx and cleaner dictionary structure
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/384
- XDR definitions works good only when both client and server talk with same XDR.
- With the extention of gluster's iatt structure there was a need to enhance the responses from server to client, so we have all these extra information.
- By bringing another version to RPC programs, the new XDR definitions are now in place.
- This uses more standard XDR for dictionary encoding and decoding along with extra iatt information. More on this is discussed on github [issue #203](https://github.com/gluster/glusterfs/issues/203)
- Release notes:
- No user impacting changes here.

#### 5. Leases support on GlusterFS
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/350
- Developer documentation is available in the header file
- Release notes:
- Can be handled by the release team

#### 6. Translator to handle 'global' options
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/303
- GlusterFS process has around 50 command line arguments to itself. While many of the options are truely initial setting, many others can change its value in volume lifetime.
- Currently there are no way to change the setting other than restarting the process for many of these options.
- The volume file structure and the translator framework can handle the case of setting and managing the options which are process specific, thus allowing us to change these options during run-time.
- This feature is much desirable for options of 'fuse' translator which, without this feature needs a remount to enable/disable.
- If you are a developer and want to provide a new command line option, consider adding it in globals.c too, and handle the option. A sample is provided at https://review.gluster.org/18316
- Release notes:
- No user impacting changes yet, as the infra is not consumed by GD2 yet.

#### 7. Information on RPC programs
**Notes for developers:**
- https://github.com/gluster/glusterfs/issues/140
- Release notes:
- https://github.com/gluster/glusterfs/blob/master/doc/developer-guide/rpc-for-glusterfs.new-versions.md

## Major issues

**TBD**

## Bugs addressed

Bugs addressed since release-3.13.0 are listed below.

**TBD**