Fix requirements listings

[brenmous]

For consideration, managing hiperseis requirements with pip-tools:

setup.py and requirements.txt are out of sync, with setup.py missing quite a few dependencies. Versions in requirements.txt are also not pinned.

To bring this in line with standards around packaging (install_requires vs requirements files), I've added the missing deps to setup.py requirements.in. I've then used pip-compile from pip-tools to generate a pinned requirements.txt file. This gives us a deterministic and replicable dev environment, while maintaining loosest working dependencies for PyPi installs.

Install behaviour

• pip install -r requirements.txt will replicate the development environment, but will require hiperseis being added to Python path
• pip install -r requirements.txt && pip install . will replicate the development environment and also install the hiperseis package into the environment
• pip install . will install the hiperseis package, fetching the loosest possible dependencies (i.e. what PyPi install would do and how non-developer users should generally install)

(Note for testing you will still need to run pip install -r tests/requirements.txt after above install - I haven't looked at synchronising this file and extras_requires.)

Using pip-tools

To use pip tools, install with pip install pip-tools. You can then use pip-compile to generate a fresh list of pinned requirements from requirements.in, which can then be installed with pip install -r requirements.txt. Add new dependencies/remove old ones from the requirements.in file (setup.py/install_requires and requirements.txt shouldn't be modified manually when using this approach) and regenerate with pip-compile when needed.

Using pip-tools to upgrade packages

pip-compile --upgrade will attempt to bump all package versions within the limits set by requirements.in - this is handy, especially if part of the CI process, as it allows us to regularly test the latest dependency versions, which gives us a heads up of new dependency versions breaking hiperseis and also getting the latest security updates, bug fixes etc.

Supporting different Python versions

Running pip-compile with a particular Python version active will generate a compatible requirements.txt for that Python version, based on what's available via pip for that Python version. Currently the committed requirements.txt was generated using Python 3.7 and is not compatible with older versions. To use an older version, such as 3.5, you would have to activate your Python 3.5 environment (with pip-tools installed) and run pip-compile to generate a new compatible dependency list. This means we need to run pip-compile as part of CI for each different Python version we build and test. It's useful in that it allows us to see the upper limits of what packages we can use given the Python version in use. Alternatively we can generate a requirements file for each version and commit them to the repo.

Using pip-tools across Linux/Windows

For packages that have sys_platform limitations, pip-compile will either need to be rerun on a Windows platform (currently compiled requirements.txt is for Linux) to generate Windows compatible packages, or alternatively we run pip-compile on Linux and Windows, committing both requirements files (e.g. windows_requirements.txt and linux_requirements.txt - see jazzband/pip-tools#585 (comment)).

[brenmous]

Need to work shapely --no-binary into the requirements.txt

[brenmous]

Above information still holds, except loose dependencies are now held in requirements.in instead of setup.py. This lets us use the features of a requirements file (e.g. --no-binary, PEP508 markers) and still use pip-tools. We then reference this in setup.py as install_requires.

In short:

• requirements.in loosest possible dependencies, will be also be read by setup.py (this is where new dependencies should be added and old ones removed)
• requirements.txt exact pinned dependencies, generated from requirements.in by pip-compile

[brenmous]

Tests are failing due to package compatibility with Python versions. Need to add compatible package versions for each supported Python version in requirements.in.

Edit: solution is to actually pip-compile and recompile the dependencies with the current Python version active, which will produce a list of compatible dependencies for that version.