# Flytekit v0.31.0

## Reminders

Please be aware of a couple things **coming down the pipe soon...**

* This is (hopefully) the penultimate release for flytekit before the 1.0 release. We have a couple major changes pending, so we plan on releasing v0.32.0 within the next few days as well. (There is a complete overhaul of flytekit's configuration system in the works, and a brand new type forthcoming (unions).)
* As of v0.32.0, we will flip the Structured Dataset [flag](https://github.com/flyteorg/flytekit/blob/457f323c0052df822c962186f44a6989a418d23d/flytekit/configuration/sdk.py#L34) to true by default. Please ensure that your backend is on Propeller version v0.16.14 and Admin version v0.6.78 or later (aka Flyte release v0.19.1 or later) otherwise you will get compiler errors if you try to mix `FlyteSchema` tasks with `pd.DataFrame` tasks. (But also, *you should upgrade* to the `StructuredDataset` type away from `FlyteSchema`.)
* We are still planning on releasing flytekit 1.0.0 in mid-April (along with all other Flyte components).

---

## Main changes

### Core
* `StructuredDataset` (and `FlyteSchema`) literals can now be cached based on a hash of the dataframe's contents, rather than the storage location. This is helpful if you have a non-cached task that produces a dataframe that a cached task consumes. With this feature, if the parent task produces the same data in the dataframe, the second task will see that as a cache hit. Also applicable to dynamic tasks, see the [issue](https://github.com/flyteorg/flyte/issues/1581) for more information.
* To turn on this feature just `typing.Annotate` with a hash function and make sure you're on Propeller v0.16.31 or later (Flyte release version v0.19.3). Documentation forthcoming.
* New [AWS Batch Task](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/aws/batch/batch.html#sphx-glr-auto-integrations-aws-batch-batch-py) type. This allows users to run a task on AWS Batch instead of on a K8s cluster.

### FlyteRemote
* `v0.30.0` added support for calling tasks and launchplans fetched from Admin via `FlyteRemote` in new `@workflow`s. This release changes behind the scenes how they work and also adds support for fetched workflows.

### Others
* We realized there was a shortcoming in the `StructuredDatasetDecoder` interface and added a field `current_task_metadata` that represents the metadata of the currently running task, since it may be more or less specific than the incoming task.
* A bunch of authentication related configuration objects have been removed since Admin has for a long time now been publishing [OAuth2 metadata](https://github.com/flyteorg/flyteidl/blob/d51d681517044d2ecdc2971466faf460fcaebb63/protos/flyteidl/service/auth.proto#L13) (and our own [metadata](https://github.com/flyteorg/flyteidl/blob/d51d681517044d2ecdc2971466faf460fcaebb63/protos/flyteidl/service/auth.proto#L49)).
* Now all you should need is:

      [platform]
      url=flyte.corp.net

since the pkce flow is assumed.
* For the client credentials flow, you just have to add:

      [credentials]
      client_id=flytepropeller
      client_secret=your.password
      auth_mode=client_credentials
      scopes=all

* The user-facing logger, (i.e. `flytekit.current_context().logging`) is now going to be a flytekit logger (see `flytekit.loggers`). Originally it was just passing through the main Python `logging` library, so if you had `basicConfig()` before you'll need to update that, or just manually switch to logging. We also swapped out flytekit internal usage of the main `logging` library in favor of specific flytekit loggers.
* New encoder/decoder in `fsspec` plugin support for `StructuredDataset`.

## Auto Generated Changelog
### What's Changed
* Assign input and output to FlyteWorkflowExecution by @pingsutw in https://github.com/flyteorg/flytekit/pull/842
* Add reference entities to FlyteTask and FlyteLaunchPlan by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/850
* Fix fast registration error by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/851
* Add support for local execute in pod task by @ggydush-fn in https://github.com/flyteorg/flytekit/pull/852
* Add anonymous retry by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/854
* [Core feature] Add Raw AWS Batch Task by @pingsutw in https://github.com/flyteorg/flytekit/pull/782
* Add structured dataset encoder/decoder in fsspec plugin by @pingsutw in https://github.com/flyteorg/flytekit/pull/849
* Delete unnecessary auth configuration by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/858
* Fixed format alias in Flytekit docs by @SmritiSatyanV in https://github.com/flyteorg/flytekit/pull/844
* Bump idl by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/862
* Updated authoring.rst by @SmritiSatyanV in https://github.com/flyteorg/flytekit/pull/863
* Updated authoring.rst by @SmritiSatyanV in https://github.com/flyteorg/flytekit/pull/866
* Change docs for HTMLPage type to say HTMLPage instead of PNGImage by @NotMatthewGriffin in https://github.com/flyteorg/flytekit/pull/868
* Revisit StructuredDatasetDecoder interface by @pingsutw in https://github.com/flyteorg/flytekit/pull/865
* Quick cleanup of contributing.rst by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/870
* Add GCS protocol in the structured dataset by @pingsutw in https://github.com/flyteorg/flytekit/pull/869
* Make fetched entities callable within workflows by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/867
* Caching of offloaded objects by @eapolinario in https://github.com/flyteorg/flytekit/pull/762
* FlyteRemote fetch of conditional nodes by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/772
* Removed root logger by @pingsutw in https://github.com/flyteorg/flytekit/pull/871
* Remove extraneous calls to set logging level by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/876
* Fix to support sphinx doctest with task decorator by @ggydush-fn in https://github.com/flyteorg/flytekit/pull/873
* [Docs] Link to data doc in FlyteFile by @wild-endeavor in https://github.com/flyteorg/flytekit/pull/877
* Fixed lint errors by @pingsutw in https://github.com/flyteorg/flytekit/pull/878

### New Contributors
* @ggydush-fn made their first contribution in https://github.com/flyteorg/flytekit/pull/852
* @SmritiSatyanV made their first contribution in https://github.com/flyteorg/flytekit/pull/844
* @NotMatthewGriffin made their first contribution in https://github.com/flyteorg/flytekit/pull/868

**Full Changelog**: https://github.com/flyteorg/flytekit/compare/v0.30.1...v0.31.0