Use pyproject.toml instead of setup.py/setup.cfg #2540

bouweandela · 2024-10-03T09:56:46Z

Description

Use pyproject.toml instead of setup.py/setup.cfg, as this is the modern way of doing things.

Also added some improvements suggested by https://learn.scientific-python.org/development/guides/repo-review/?repo=esmvalgroup%2Fesmvalcore&branch=pyproject-toml

Link to documentation: https://esmvaltool--2540.org.readthedocs.build/projects/ESMValCore/en/2540/contributing.html

Related to ESMValGroup/ESMValTool#3584

Before you get started

☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

🧪 The new functionality is relevant and scientifically sound
🛠 This pull request has a descriptive title and labels
🧪 and 🛠 Documentation is available
🛠 Changes are backward compatible
🛠 The list of authors is up to date
🛠 All checks below this pull request were successful - except that a line that has been reformatted by ruff (probably because we now specify the required python version) is not covered by tests

To help with the number pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

codecov · 2024-10-03T09:57:07Z

Codecov Report

Attention: Patch coverage is 50.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 94.67%. Comparing base (a3557ec) to head (1a7c867).

Files with missing lines	Patch %	Lines
esmvalcore/_task.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2540      +/-   ##
==========================================
- Coverage   94.91%   94.67%   -0.25%     
==========================================
  Files         251      251              
  Lines       14268    14266       -2     
==========================================
- Hits        13543    13506      -37     
- Misses        725      760      +35

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

valeriupredoi

absolutely YES for this 🍺

valeriupredoi · 2024-10-03T10:42:08Z

setup.cfg

-    pragma: no cover
-    if __name__ == .__main__.:
-    if TYPE_CHECKING:
-
 [pycodestyle]


love the outcome of this issue, that was proposing support for pyproject.toml 🤣 PyCQA/pycodestyle#813

We should be able to drop pycodestyle as soon as we're using ruff for linting too.

…ct-toml

schlunma

Great, thanks Bouwe, I really like this! 🚀

There's one more suggestion at https://learn.scientific-python.org/development/guides/repo-review/?repo=esmvalgroup%2Fesmvalcore&branch=pyproject-toml about pyproject:

[PP309](https://learn.scientific-python.org/development/guides/pytest#PP309): Filter warnings specified

filterwarnings must be set (probably to at least ["error"]). Python will hide important warnings otherwise, like deprecations.

[tool.pytest.ini_options]
filterwarnings = ["error"]

Would it make sense to implement this as well?

schlunma · 2024-10-15T14:48:00Z

.circleci/install_triggers

@@ -1,5 +1,3 @@
 ^\.circleci/
 ^environment\.yml$
 ^pyproject.toml$
-^setup\.py$
-^setup\.cfg$


setup.cfg still exists, did you remove that here on purpose?

Yes, it now only contains the configuration for pycodestyle, so it does not affect the (development) installation of the tool anymore.

Makes sense. Would it make sense to remove all unused element in setup.cfg here? It looks like it contains more than the pycodestyle config:

ESMValCore/setup.cfg

Lines 1 to 38 in a3557ec

[tool:pytest]

addopts =

--doctest-modules

--ignore=esmvalcore/cmor/tables/

--cov-report=xml:test-reports/coverage.xml

--cov-report=html:test-reports/coverage_html

--html=test-reports/report.html

env =

MPLBACKEND = Agg

log_level = WARNING

markers =

installation: Test requires installation of dependencies

use_sample_data: Run functional tests using real data

[coverage:run]

parallel = true

source = esmvalcore

[coverage:report]

exclude_lines =

pragma: no cover

if __name__ == .__main__.:

if TYPE_CHECKING:

[pycodestyle]

# ignore rules that conflict with ruff formatter

# E203: https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#slices

# E501: https://docs.astral.sh/ruff/formatter/#conflicting-lint-rules

# W503: https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes

ignore = E203,E501,W503

[pydocstyle]

convention = numpy

[mypy]

# see mypy.readthedocs.io/en/stable/command_line.html

python_version = 3.12

ignore_missing_imports = True

files = esmvalcore, tests

Looks like I messed up the last merge with main. Should be fixed now.

And then I forgot to move the changes from #2456 to pyproject.toml. Should be really fixed now 🤦

schlunma · 2024-10-15T14:56:04Z

tests/integration/cmor/_fixes/cmip6/test_cesm2.py

@@ -61,7 +61,6 @@ def test_get_cl_fix():
 )


-@pytest.mark.sequential
 @pytest.mark.skipif(
    sys.version_info < (3, 7, 6), reason="requires python3.7.6 or newer"
 )


Why are you dropping the sequential mark? Also, would it make sense to remove the following skipif since we're not supporting Python 3.7 anymore?

It was removed in #1819, but these entries were forgotten. Using

ESMValCore/pyproject.toml

Line 151 in edb1291

"--strict-markers",

makes using undefined markers an error.

Also, would it make sense to remove the following skipif since we're not supporting Python 3.7 anymore?

Removed in e70ffd2

bouweandela · 2024-10-15T15:27:58Z

Would it make sense to implement this as well?

I tried, but it causes the tests to fail because some of our dependencies are using deprecated features. It would be good to enable that though, but it will require extra work.

bouweandela · 2024-10-15T18:18:24Z

Would it make sense to implement this as well?

I tried, but it causes the tests to fail because some of our dependencies are using deprecated features. It would be good to enable that though, but it will require extra work.

Created #2555 as a reminder.

…ct-toml

bouweandela added the installation Installation problem label Oct 3, 2024

valeriupredoi reviewed Oct 3, 2024

View reviewed changes

bouweandela added 4 commits October 3, 2024 13:09

Use pyproject.toml

6874109

Fix formatting issue

7eda427

Use recommendations

ed28019

Enable extra check

8e922f6

bouweandela force-pushed the pyproject-toml branch from f25c4f1 to 8e922f6 Compare October 3, 2024 11:09

bouweandela marked this pull request as ready for review October 3, 2024 12:39

bouweandela added 4 commits October 10, 2024 15:47

Merge branch 'main' of github.com:ESMValGroup/ESMValCore into pyproje…

29cc323

…ct-toml

Remove setup.py and setup.cfg from list of installation files

8e9204a

Merge branch 'main' of github.com:ESMValGroup/ESMValCore into pyproje…

dbbd064

…ct-toml

Merge branch 'main' into pyproject-toml

b0ff2d0

schlunma reviewed Oct 15, 2024

View reviewed changes

bouweandela force-pushed the pyproject-toml branch from edb1291 to 8b37c2d Compare October 15, 2024 17:58

bouweandela added 2 commits October 16, 2024 09:12

Merge branch 'main' of github.com:ESMValGroup/ESMValCore into pyproje…

7451ce2

…ct-toml

Remove checks for unsupported Python versions

1a7c867

bouweandela force-pushed the pyproject-toml branch from e70ffd2 to 1a7c867 Compare October 16, 2024 07:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use pyproject.toml instead of setup.py/setup.cfg #2540

Use pyproject.toml instead of setup.py/setup.cfg #2540

bouweandela commented Oct 3, 2024 •

edited

Loading

codecov bot commented Oct 3, 2024 •

edited

Loading

valeriupredoi left a comment

valeriupredoi Oct 3, 2024

bouweandela Oct 3, 2024

schlunma left a comment

schlunma Oct 15, 2024

bouweandela Oct 15, 2024

schlunma Oct 15, 2024

bouweandela Oct 15, 2024

bouweandela Oct 16, 2024

schlunma Oct 15, 2024

bouweandela Oct 15, 2024

bouweandela Oct 15, 2024 •

edited

Loading

bouweandela commented Oct 15, 2024

bouweandela commented Oct 15, 2024 •

edited

Loading

	[tool:pytest]
	addopts =
	--doctest-modules
	--ignore=esmvalcore/cmor/tables/
	--cov-report=xml:test-reports/coverage.xml
	--cov-report=html:test-reports/coverage_html
	--html=test-reports/report.html
	env =
	MPLBACKEND = Agg
	log_level = WARNING
	markers =
	installation: Test requires installation of dependencies
	use_sample_data: Run functional tests using real data

	[coverage:run]
	parallel = true
	source = esmvalcore
	[coverage:report]
	exclude_lines =
	pragma: no cover
	if __name__ == .__main__.:
	if TYPE_CHECKING:

	[pycodestyle]
	# ignore rules that conflict with ruff formatter
	# E203: https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#slices
	# E501: https://docs.astral.sh/ruff/formatter/#conflicting-lint-rules
	# W503: https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes
	ignore = E203,E501,W503

	[pydocstyle]
	convention = numpy

	[mypy]
	# see mypy.readthedocs.io/en/stable/command_line.html
	python_version = 3.12
	ignore_missing_imports = True
	files = esmvalcore, tests

Use pyproject.toml instead of setup.py/setup.cfg #2540

Are you sure you want to change the base?

Use pyproject.toml instead of setup.py/setup.cfg #2540

Conversation

bouweandela commented Oct 3, 2024 • edited Loading

Description

codecov bot commented Oct 3, 2024 • edited Loading

Codecov Report

valeriupredoi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

schlunma left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bouweandela Oct 15, 2024 • edited Loading

Choose a reason for hiding this comment

bouweandela commented Oct 15, 2024

bouweandela commented Oct 15, 2024 • edited Loading

bouweandela commented Oct 3, 2024 •

edited

Loading

codecov bot commented Oct 3, 2024 •

edited

Loading

bouweandela Oct 15, 2024 •

edited

Loading

bouweandela commented Oct 15, 2024 •

edited

Loading