Run all pre-checks in a single job

[javitonino]

Description

• Move all pre-checks to the New CI pipeline in a single job (less workers taken for linting = more workers available for testing)
• Does a single lint for nucliadb (not per-component)
• Switches flake8 to ruff (only for flake checks, formatting is still done with isort+black to minimize reformatting). The reason for this change is that ruff allows specifying a target python version for checks, so we can use a single Python installation (3.12) for checking all versions since 3.9.
• Removes code-requirements.txt, these are all managed by pdm now.
• Removes a bunch of ignored flake8 rules and standardizes in a single set (in ruff.toml). Fixes a few instances of rules being broken.

[github-actions]

Benchmark

Benchmark suite	Current: e3f482b	Previous: 0d03d9f	Ratio
tests/search/unit/search/test_fetch.py::test_highligh_error	2836.842899803532 iter/sec (stddev: 0.000009108893831615426)	2841.0684406726436 iter/sec (stddev: 0.000004954958228416619)	1.00

This comment was automatically generated by workflow using github-action-benchmark.