0.19.9

Major features and improvements

• Dropped Python 3.8 support.
• Implemented KedroDataCatalog repeating DataCatalog functionality with a few API enhancements:
  • Removed _FrozenDatasets and access datasets as properties;
  • Added get dataset by name feature;
  • add_feed_dict() was simplified to only add raw data;
  • Datasets' initialisation was moved out from from_config() method to the constructor.
• Moved development requirements from requirements.txt to the dedicated section in pyproject.toml for project template.
• Implemented Protocol abstraction for the current DataCatalog and adding new catalog implementations.
• Refactored kedro run and kedro catalog commands.
• Moved pattern resolution logic from DataCatalog to a separate component - CatalogConfigResolver. Updated DataCatalog to use CatalogConfigResolver internally.
• Made packaged Kedro projects return session.run() output to be used when running it in the interactive environment.
• Enhanced OmegaConfigLoader configuration validation to detect duplicate keys at all parameter levels, ensuring comprehensive nested key checking.

Note: KedroDataCatalog is an experimental feature and is under active development. Therefore, it is possible we'll introduce breaking changes to this class, so be mindful of that if you decide to use it already. Let us know if you have any feedback about the KedroDataCatalog or ideas for new features.

Bug fixes and other changes

• Fixed bug where using dataset factories breaks with ThreadRunner.
• Fixed a bug where SharedMemoryDataset.exists would not call the underlying MemoryDataset.
• Fixed template projects example tests.
• Made credentials loading consistent between KedroContext._get_catalog() and resolve_patterns so that both use _get_config_credentials()

Breaking changes to the API

• Removed ShelveStore to address a security vulnerability.

Documentation changes

• Fix logo on PyPI page.
• Minor language/styling updates.

Community contributions

• Puneet
• ethanknights
• Manezki
• MigQ2
• Felix Scherz
• Yu-Sheng Li

0.19.8

Major features and improvements

• Made default run entrypoint in __main__.py work in interactive environments such as IPyhon and Databricks.

Bug fixes and other changes

• Fixed a bug that caused tracebacks disappeared from CLI runs.
• Moved _find_run_command() and _find_run_command_in_plugins() from __main__.py in the project template to the framework itself.
• Fixed a bug where %load_node breaks with multi-lines import statements.
• Fixed a regression where rich mark up logs stop showing since 0.19.7.

Breaking changes to the API

Documentation changes

• Add clarifications in docs explaining how runtime parameter resolution works.

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

• cclauss
• eltociear
• ltalirz

0.19.7

Major features and improvements

• Exposed load and save publicly for each dataset in the core kedro library, and enabled other datasets to do the same. If a dataset doesn't expose load or save publicly, Kedro will fall back to using _load or _save, respectively.
• Kedro commands are now lazily loaded to add performance gains when running Kedro commands.
• Implemented key completion support for accessing datasets in the DataCatalog.
• Implemented dataset pretty printing.
• Implemented DataCatalog pretty printing.
• Moved to an opt-out model for telemetry, enabling it by default without requiring prior consent.

Bug fixes and other changes

• Updated error message for invalid catalog entries.
• Updated error message for catalog entries when the dataset class is not found with hints on how to resolve the issue.
• Fixed a bug in the DataCatalog shallow_copy() method to ensure it returns the type of the used catalog and doesn't cast it to DataCatalog.
• Made kedro-telemetry a core dependency.
• Fixed a bug when OmegaConfigLoader is printed, there are few missing arguments.
• Fixed a bug when where iterating OmegaConfigLoader's keys return empty dictionary.

Breaking changes to the API

Upcoming deprecations for Kedro 0.20.0

• The utility method get_pkg_version() is deprecated and will be removed in Kedro 0.20.0.

Documentation changes

• Improved documentation for configuring dataset parameters in the data catalog
• Extended documentation with an example of logging customisation at runtime

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

• nickolasrm
• yury-fedotov

0.19.6

Major features and improvements

• Added raise_errors argument to find_pipelines. If True, the first pipeline for which autodiscovery fails will cause an error to be raised. The default behaviour is still to raise a warning for each failing pipeline.
• It is now possible to use Kedro without having rich installed.
• Updated custom logging behavior: conf/logging.yml will be used if it exists and KEDRO_LOGGING_CONFIG is not set; otherwise, default_logging.yml will be used.

Bug fixes and other changes

• User defined catch-all dataset factory patterns now override the default pattern provided by the runner.

Breaking changes to the API

Upcoming deprecations for Kedro 0.20.0

• All micro-packaging commands (kedro micropkg pull, kedro micropkg package) are deprecated and will be removed in Kedro 0.20.0.

Documentation changes

• Improved documentation for custom starters
• Added a new docs section on deploying Kedro project on AWS Airflow MWAA
• Detailed instructions on using globals and runtime_params with the OmegaConfigLoader

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

• doxenix
• cleeeks

0.19.5

Bug fixes and other changes

• Fixed breaking import issue when working on a project with kedro-viz on python 3.8.

Documentation changes

• Updated the documentation for deploying a Kedro project with Astronomer Airflow.
• Used kedro-sphinx-theme for documentation.

0.19.4

Major features and improvements

• Kedro commands now work from any subdirectory within a Kedro project.
• Kedro CLI now provides a better error message when project commands are run outside of a project i.e. kedro run
• Added the --telemetry flag to kedro new, allowing the user to register consent to have user analytics collected at the same time as the project is created.
• Improved the performance of Pipeline object creation and summing.
• Improved suggestions to resume failed pipeline runs.
• Dropped the dependency on toposort in favour of the built-in graphlib module.
• Cookiecutter errors are shown in short format without the --verbose flag.

Bug fixes and other changes

• Updated kedro pipeline create and kedro pipeline delete to read the base environment from the project settings.
• Updated CLI command kedro catalog resolve to read credentials properly.
• Changed the path of where pipeline tests generated with kedro pipeline create from <project root>/src/tests/pipelines/<pipeline name> to <project root>/tests/pipelines/<pipeline name>.
• Updated .gitignore to prevent pushing Mlflow local runs folder to a remote forge when using mlflow and git.
• Fixed error handling message for malformed yaml/json files in OmegaConfigLoader.
• Fixed a bug in node-creation allowing self-dependencies when using transcoding, that is datasets named like name@format.
• Improved error message when passing wrong value to node.

Breaking changes to the API

• Methods _is_project and _find_kedro_project have been moved to kedro.utils. We recommend not using private methods in your code, but if you do, please update your code to use the new location.

Documentation changes

• Added missing description for merge_strategy argument in OmegaConfigLoader.
• Added documentation on best practices for testing nodes and pipelines.
• Clarified docs around using custom resolvers without a full Kedro project.

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

• ondrejzacha
• Puneet

0.19.3

Major features and improvements

• Create the debugging line magic %load_node for Jupyter Notebook and Jupyter Lab.
• Add better IPython, VSCode Notebook support for %load_node and minimal support for Databricks.
• Add full Kedro Node input syntax for %load_node.

Bug fixes and other changes

• Updated CLI Command kedro catalog resolve to work with dataset factories that use PartitionedDataset.
• Addressed arbitrary file write via archive extraction security vulnerability in micropackaging.
• Added the _EPHEMERAL attribute to AbstractDataset and other Dataset classes that inherit from it.
• Added new JSON Schema that works with Kedro versions 0.19.*

Breaking changes to the API

Documentation changes

• Enable read-the-docs search when user presses Command/Ctrl + K.
• Added documentation for kedro-telemetry and the data collected by it.

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

• MosaicMan
• Fazil

0.19.2

Bug fixes and other changes

• Removed example pipeline requirements when examples are not selected in tools.
• Allowed modern versions of JupyterLab and Jupyter Notebooks.
• Removed setuptools dependency
• Added source_dir explicitly in pyproject.toml for non-src layout project.
• MemoryDataset entries are now included in free outputs.
• Removed black dependency and replaced it functionality with ruff format.
• Added logging about not using async mode in SequentiallRunner and ParallelRunner.

Breaking changes to the API

• Changed input format for tools option obtained from --config file from numbers to short names.

Documentation changes

• Added documentation about bootstrap_project and configure_project.
• Added documentation about kedro run and hook execution order.

0.19.1

Release 0.19.1

What's Changed

• Loosen bound for kedro-telemetry by @merelcht in #3417

0.19.0

🚀 Major Features and improvements

• Dropped Python 3.7 support.
• Introduced project tools and example to the kedro new CLI flow.
• The new spaceflights starters, spaceflights-pandas, spaceflights-pandas-viz, spaceflights-pyspark, and spaceflights-pyspark-viz can be used with the kedro new command with the --starter flag.
• Added the --conf-source option to %reload_kedro, allowing users to specify a source for project configuration.
• Added the functionality to choose a merging strategy for config files loaded with OmegaConfigLoader.
• Modified the mechanism of importing datasets, raise more explicit error when dependencies are missing.
• Added validation for configuration file used to override run commands via the CLI.
• Moved the default environment base and local from config loader to _ProjectSettings. This enables the use of config loader as a standalone class without affecting existing Kedro Framework users.

🪲 Bug fixes and other changes

• Added a new field tools to pyproject.toml when a project is created.
• Reduced spaceflights data to minimise waiting times during tutorial execution.
• Added validation to node tags to be consistent with node names.
• Removed pip-tools as a dependency.
• Accepted path-like filepaths more broadly for datasets.

💥 Breaking changes

• Removed ConfigLoader and TemplatedConfigLoader.
• Removed kedro.extras.datasets and tests (use kedro-datasets instead)
• Removed PartitionedDataset and IncrementalDataset from kedro.io (import them from kedro-datasets instead)
• logging is removed from OmegaConfigLoader in favour of the environment variable KEDRO_LOGGING_CONFIG.
• Removed support for defining the layer attribute at top-level within DataCatalog.
• Renamed data_set and DataSet to dataset and Dataset everywhere.
• Removed the create_default_data_set() method in the Runner in favour of using dataset factories to create default dataset instances.
• The default project template now has only one pyproject.toml at the root of the project (containing both the packaging metadata and the Kedro build config).

✍️ Documentation changes

• Added new top navigation to easily switch between Framework, Viz, and Datasets.
• Added new search-as-you-type to improve the search experience.

New Contributors

• @MinuraPunchihewa made their first contribution in #3115
• @mustious made their first contribution in #3181
• @JayOaks made their first contribution in #3239
• @adamkells made their first contribution in #3203
• @HKABIG made their first contribution in #3270
• @pdave34 made their first contribution in #3213
• @hermlon made their first contribution in #3303

Full Changelog: 0.18.14...0.19.0

🚨 If you are upgrading from Kedro 0.18, have a look at the migration guide.

We welcome every community contribution, large or small. See what we're working on now and report bugs or suggest future features.
Until next time,
The Kedro Team 💛