Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SampleSupervisor #101

Open
wants to merge 99 commits into
base: v3.2
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
731910e
feat: added SampleSupervisor
rabii-chaarani Jun 3, 2024
42a6238
fix: added StateType enum
rabii-chaarani Jun 3, 2024
8e6652d
doc: added libtiff-dev instead of libtiff5
rabii-chaarani Jun 3, 2024
aca1164
fix: updated to align SampleSupervisor
rabii-chaarani Jun 3, 2024
439c383
fix: fault_obs_data aligned to master branch
rabii-chaarani Jun 3, 2024
47a0b69
test: updated InterpolatedStructure test
rabii-chaarani Jun 3, 2024
cff5d39
test: add m2l import
rabii-chaarani Jun 3, 2024
06e6b10
test: changed test data
rabii-chaarani Jun 3, 2024
f6618cf
test: fixed structure config for test
rabii-chaarani Jun 3, 2024
4be78bc
fix: inplace to false
rabii-chaarani Jun 3, 2024
d425b3c
fix: fixed set_crs method
rabii-chaarani Jun 3, 2024
09dd7f3
fix: ensure all dipdir vals < 360
AngRodrigues May 30, 2024
a37058b
docs: update libtiff version
AngRodrigues May 30, 2024
d3fc6b5
fix: add test for all structures less than 360
AngRodrigues May 30, 2024
9efe63b
fix: add catchall exception to mapdata_parse_structure & comment code
AngRodrigues May 30, 2024
1aa5add
fix: make the templates a bit easier to fill out
AngRodrigues May 30, 2024
c61e225
fix: update question template
AngRodrigues May 30, 2024
8be28c8
fix: move location of the PR template
AngRodrigues May 30, 2024
f7b29c6
fix: fix for altitude not being saved properly in LPF
AngRodrigues May 30, 2024
6efdc98
fix: update tests that depend on server to skip in case server is una…
AngRodrigues May 31, 2024
933130e
test: fix for when server not available
AngRodrigues May 31, 2024
b4fba56
fix: allow 2 minutes for server connection & add add available ga ser…
AngRodrigues May 31, 2024
78c5a80
fix: add 2 more links to try from GA WCS if timing out & prints out w…
AngRodrigues May 31, 2024
4a05764
fix: revert back to original
AngRodrigues May 31, 2024
024f67f
fix: create the tmp file function missing
AngRodrigues May 31, 2024
f6fe370
fix: return just column with new vals instead of inplace modification
AngRodrigues Jun 3, 2024
7d9e6d7
fix: use gpd.points_from_xy
AngRodrigues Jun 3, 2024
f7fd8e8
fix: verbose dipdir 360 test name
AngRodrigues Jun 3, 2024
9285be5
fix: revert back to original timeout
AngRodrigues Jun 6, 2024
1ade12d
fix: revert back to original url
AngRodrigues Jun 6, 2024
ca5cb0b
chore(master): release 3.1.5
github-actions[bot] Jun 6, 2024
c323df5
test: changed STRIKE to DIPDIR
rabii-chaarani Jun 7, 2024
2def0a8
fix: Added DIPDIR to dipdir_column
rabii-chaarani Jun 7, 2024
a032a24
test: samples return sample_supervisor
rabii-chaarani Jun 7, 2024
74a35cd
fix: remove sample processing from Project class
rabii-chaarani Jun 7, 2024
47d51f3
test: fixed data error in test
rabii-chaarani Jun 7, 2024
8d7e44e
test: use SampleSupervisor to set sampler
rabii-chaarani Jun 7, 2024
a9a3a7b
test: use correct type
rabii-chaarani Jun 7, 2024
138ae15
test: corrected fixture name
rabii-chaarani Jun 7, 2024
b288446
test: use correct dipdir column
rabii-chaarani Jun 7, 2024
f31c54a
test: changed dip column name
rabii-chaarani Jun 7, 2024
e1d5c25
test: updated example date used in test
rabii-chaarani Jun 7, 2024
692864a
fix: use correct type for samplers
rabii-chaarani Jun 7, 2024
569e6ee
fix: update dependencies as discussed
AngRodrigues May 24, 2024
bf61bc4
Update issue templates
lachlangrose May 20, 2024
980f24f
feat: added issue_templates
AngRodrigues May 22, 2024
81a9a02
fix: add pull request template
AngRodrigues May 23, 2024
bf129e8
fix: few typos grammar
AngRodrigues May 23, 2024
fd567fc
fix: tests should be with pytest
AngRodrigues May 23, 2024
523ee4b
Revert "Update issue templates"
AngRodrigues May 27, 2024
7929469
fix: trigger build
lachlangrose May 9, 2024
131b89a
style: style fixes by ruff and autoformatting by black
lachlangrose May 9, 2024
0b38a46
Update issue templates
lachlangrose May 20, 2024
ec6a020
feat: added issue_templates
AngRodrigues May 22, 2024
ff54560
fix: add pull request template
AngRodrigues May 23, 2024
6f787b9
fix: few typos grammar
AngRodrigues May 23, 2024
84764ce
fix: tests should be with pytest
AngRodrigues May 23, 2024
28f437b
Revert "Update issue templates"
AngRodrigues May 27, 2024
3bfffb8
fix: add issue templates back
AngRodrigues May 28, 2024
7605c77
fix: correct grammar for clarity
AngRodrigues May 28, 2024
f89d9dc
Merge branch 'v3.2' into sample_storage_3
rabii-chaarani Jun 12, 2024
00cea8f
fix: avoid double imports
AngRodrigues May 29, 2024
ae11ebe
fix: add test for all structures less than 360
AngRodrigues May 30, 2024
e94dceb
fix: add catchall exception to mapdata_parse_structure & comment code
AngRodrigues May 30, 2024
7e25622
fix: make the templates a bit easier to fill out
AngRodrigues May 30, 2024
7dff3b4
fix: update question template
AngRodrigues May 30, 2024
d210914
fix: fix for altitude not being saved properly in LPF
AngRodrigues May 30, 2024
9f7ddab
fix: allow 2 minutes for server connection & add add available ga ser…
AngRodrigues May 31, 2024
02a6835
fix: add 2 more links to try from GA WCS if timing out & prints out w…
AngRodrigues May 31, 2024
ccee817
fix: revert back to original
AngRodrigues May 31, 2024
17aa2b3
fix: create the tmp file function missing
AngRodrigues May 31, 2024
dac2a90
fix: return just column with new vals instead of inplace modification
AngRodrigues Jun 3, 2024
4e042e5
fix: verbose dipdir 360 test name
AngRodrigues Jun 3, 2024
3978108
fix: revert back to original timeout
AngRodrigues Jun 6, 2024
9a19373
fix: revert back to original url
AngRodrigues Jun 6, 2024
11d507f
fix: remove matplotlib & assign random colors & tidy up imports
AngRodrigues May 30, 2024
6a8197d
style: style fixes by ruff and autoformatting by black
AngRodrigues May 30, 2024
51f878d
fix: assign the right beartype output in create_points
AngRodrigues May 31, 2024
437e47d
fix: add invalid hex input handling
AngRodrigues Jun 6, 2024
27e14a8
fix: comment the code
AngRodrigues Jun 6, 2024
c1fcfda
tests: add test for mapdata.colour_units
AngRodrigues Jun 6, 2024
072bb34
tests: reorganise tests
AngRodrigues Jun 6, 2024
c2c8f1e
fix: make sure all colours are unique
AngRodrigues Jun 6, 2024
40b3042
fix: small fixes - double check for integer & add test information at…
AngRodrigues Jun 7, 2024
049f00c
fix: fix for duplicate units
AngRodrigues Jun 7, 2024
9e63732
tests: add tests for duplicate units in colour_units()
AngRodrigues Jun 7, 2024
afe4d5a
tests: rephrase the unit must be unique assertion in colour_unit_tests
AngRodrigues Jun 7, 2024
5c87e5d
fix: add the suggestions
AngRodrigues Jun 13, 2024
fad1f3c
fix: update the tests for new function names (hex_to_rgb)
AngRodrigues Jun 13, 2024
d5726f3
chore(master): release 3.1.6
github-actions[bot] Jun 13, 2024
975863d
feat: added SampleSupervisor
rabii-chaarani Jun 3, 2024
eb42ed5
feat: added issue_templates
AngRodrigues May 22, 2024
ca40991
fix: add pull request template
AngRodrigues May 23, 2024
e9d3b8b
fix: few typos grammar
AngRodrigues May 23, 2024
eeddac3
Revert "Update issue templates"
AngRodrigues May 27, 2024
1eff115
feat: added issue_templates
AngRodrigues May 22, 2024
78e5c58
fix: add issue templates back
AngRodrigues May 28, 2024
80826a7
fix: correct grammar for clarity
AngRodrigues May 28, 2024
4f299b9
fix: add 2 more links to try from GA WCS if timing out & prints out w…
AngRodrigues May 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 9 additions & 10 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,13 @@ body:
## Bug Report
Thanks for submitting a bug report to map2loop!
Please use this template to report a bug. Please provide as much detail as possible to help us reproduce and fix the issue efficiently.
- type: input
id: bug_title

- type: textarea
id: version
attributes:
label: "Bug Title"
description: "Provide a concise and descriptive title for the bug report."
placeholder: "Enter the title of the bug"
validations:
required: true
label: Version
description: What version of map2loop and LoopProjectFile are you running? You can find this information by running `import map2loop` and `map2loop.__version__` in your python terminal or jupyter notebook.
placeholder: "Enter map2loop and LoopProjectFile versions"

- type: textarea
id: bug_description
Expand All @@ -32,7 +30,7 @@ body:
id: steps_to_reproduce
attributes:
label: "Minimal reproducible example"
description: "Provide a minimal reproducible example with the code and data necessary to reproduce the bug."
description: "Provide a minimal reproducible example with the code necessary to reproduce the bug. For more guidance, visit: [How to create a minimal complete reproducible example](https://forum.access-hive.org.au/t/how-to-create-a-minimal-complete-reproducible-example/843)"
placeholder: "Enter the steps to reproduce the bug"
validations:
required: true
Expand Down Expand Up @@ -84,4 +82,5 @@ body:
- label: "High"
- label: "Critical"
validations:
required: true
required: true

10 changes: 1 addition & 9 deletions .github/ISSUE_TEMPLATE/documentation_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,7 @@ body:
value: |
## Documentation Request
Please use this template to suggest an improvement or addition to map2loop documentation.
Provide as much detail as possible to help us understand and implement your request efficiently.
- type: input
id: doc_title
attributes:
label: "Documentation Title"
description: "Provide a concise and descriptive title for the documentation request."
placeholder: "Enter the title of the documentation request"
validations:
required: true
Provide as much detail as possible to help us understand and implement your request efficiently

- type: textarea
id: doc_details
Expand Down
23 changes: 16 additions & 7 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,16 @@ body:
value: |
## Feature Request
Please use this template to submit your feature request. Provide as much detail as possible to help us understand and implement your request efficiently.
- type: input
id: feature_title

- type: checkboxes
id: input1
attributes:
label: "Feature Title"
description: "Provide a concise and descriptive title for the feature request."
placeholder: "Enter the title of the feature"
validations:
required: true
label: "💻"
description: |
Check this if you would like to try and implement your vision in a PR.
The map2loop team will help you go through the process
options:
- label: Would you like to work on this feature?

- type: textarea
id: feature_description
Expand All @@ -35,6 +37,13 @@ body:
validations:
required: true

- type: textarea
id: version
attributes:
label: Version
description: What version of map2loop and LoopProjectFile are you running that doesn't have this feature? You can find this information by running `import map2loop` and `map2loop.__version__` in your python terminal or jupyter notebook.
placeholder: "Enter map2loop and LoopProjectFile versions"

- type: textarea
id: proposed_solution
attributes:
Expand Down
8 changes: 0 additions & 8 deletions .github/ISSUE_TEMPLATE/question.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,6 @@ body:
## Question
Please use this template to ask a question about applying map2loop to your data.
Provide as much detail as possible to help us understand and answer your question efficiently.
- type: input
id: question_title
attributes:
label: "Question Title"
description: "Provide a concise and descriptive title for your question."
placeholder: "Enter the title of your question"
validations:
required: true

- type: textarea
id: question_details
Expand Down
29 changes: 29 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,34 @@
# Changelog

## [3.1.5](https://github.com/Loop3D/map2loop/compare/v3.1.4...v3.1.5) (2024-06-06)


### Bug Fixes

* add 2 more links to try from GA WCS if timing out & prints out where the DEM was downloaded from ([92f73a5](https://github.com/Loop3D/map2loop/commit/92f73a55ad998af76fe02fd09702d98492c6e431))
* add catchall exception to mapdata_parse_structure & comment code ([59c677c](https://github.com/Loop3D/map2loop/commit/59c677c5988c1e411723c61f3482dbd792ed19a7))
* add test for all structures less than 360 ([f08c42a](https://github.com/Loop3D/map2loop/commit/f08c42a7e455657c0d042e1346fa5ce5fdb98775))
* allow 2 minutes for server connection & add add available ga server link ([900a50d](https://github.com/Loop3D/map2loop/commit/900a50d34f44e50c508466d617e4ce8d85c0c316))
* avoid double imports ([3347751](https://github.com/Loop3D/map2loop/commit/334775120d8db88a82e70980f68940d024524687))
* create the tmp file function missing ([55688fe](https://github.com/Loop3D/map2loop/commit/55688fe85f7aca2678a15160a73a972427a3bf34))
* ensure all dipdir vals &lt; 360 ([cf21a6b](https://github.com/Loop3D/map2loop/commit/cf21a6ba8cc0c48dfb7cc442d59a6cd63678ab83))
* fix for altitude not being saved properly in LPF ([b2c6638](https://github.com/Loop3D/map2loop/commit/b2c663866f478752670aca41a188aa195df9c90f))
* make the templates a bit easier to fill out ([81f54fe](https://github.com/Loop3D/map2loop/commit/81f54fe9f7163dc997219b4fe092c81b6c7b3ca3))
* move location of the PR template ([9209c25](https://github.com/Loop3D/map2loop/commit/9209c251520bfd993a51b04124ad62a831a58922))
* return just column with new vals instead of inplace modification ([722b98c](https://github.com/Loop3D/map2loop/commit/722b98cdbcf1f820b9d1a480f09ac59d287f1a04))
* revert back to original ([6ed00c9](https://github.com/Loop3D/map2loop/commit/6ed00c999901c3236e53fc039930510be9d38570))
* revert back to original timeout ([301242f](https://github.com/Loop3D/map2loop/commit/301242f931b92ac01c4251ecf9039119d9da49d5))
* revert back to original url ([26e4971](https://github.com/Loop3D/map2loop/commit/26e497198744c591879fea96c10935d6b79b7d9d))
* update question template ([8182e50](https://github.com/Loop3D/map2loop/commit/8182e50361852de817e1440adcb278b5f46f4796))
* update tests that depend on server to skip in case server is unavailable ([646be9e](https://github.com/Loop3D/map2loop/commit/646be9ea06dade92f16208da7e681d6d2b0e6c65))
* use gpd.points_from_xy ([2f931c5](https://github.com/Loop3D/map2loop/commit/2f931c59025997133aeea9c10f725858613b7f6b))
* verbose dipdir 360 test name ([6ffe6bf](https://github.com/Loop3D/map2loop/commit/6ffe6bf31ab860ee054bb175a63c00fee7b8a7ff))


### Documentation

* update libtiff version ([0a99ac8](https://github.com/Loop3D/map2loop/commit/0a99ac8aaf6980196216137f15060c44b2f77cd9))

## [3.1.4](https://github.com/Loop3D/map2loop/compare/v3.1.3...v3.1.4) (2024-05-29)


Expand Down
2 changes: 1 addition & 1 deletion docs/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ RUN apt-get update -qq && \
make\
libgl1\
libtinfo5\
libtiff5\
libtiff6\
libgl1-mesa-glx


Expand Down
6 changes: 3 additions & 3 deletions docs/examples/plot_hamersley.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"""

from map2loop.project import Project
from map2loop.m2l_enums import VerboseLevel, Datatype
from map2loop.m2l_enums import VerboseLevel, Datatype, SampleType
from map2loop.sorter import SorterAlpha
from map2loop.sampler import SamplerSpacing

Expand Down Expand Up @@ -36,8 +36,8 @@
)

# Set the distance between sample points for arial and linestring geometry
proj.set_sampler(Datatype.GEOLOGY, SamplerSpacing(200.0))
proj.set_sampler(Datatype.FAULT, SamplerSpacing(200.0))
proj.sample_supervisor.set_sampler(SampleType.GEOLOGY, SamplerSpacing(200.0))
proj.sample_supervisor.set_sampler(SampleType.FAULT, SamplerSpacing(200.0))

# Choose which stratigraphic sorter to use or run_all with "take_best" flag to run them all
proj.set_sorter(SorterAlpha())
Expand Down
14 changes: 14 additions & 0 deletions map2loop/m2l_enums.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
from enum import IntEnum

class SampleType(IntEnum):
GEOLOGY = 0
STRUCTURE = 1
FAULT = 2
FOLD = 3
FAULT_ORIENTATION = 4
CONTACT = 5
DTM = 6


class Datatype(IntEnum):
GEOLOGY = 0
Expand Down Expand Up @@ -30,3 +39,8 @@ class VerboseLevel(IntEnum):
NONE = 0
TEXTONLY = 1
ALL = 2


class StateType(IntEnum):
DATA = 0
SAMPLER = 1
20 changes: 17 additions & 3 deletions map2loop/mapdata.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
from .aus_state_urls import AustraliaStateUrls
from .random_colour import random_colours_hex
from typing import Union

import requests
import requests

class MapData:
"""
Expand Down Expand Up @@ -482,6 +483,8 @@ def __check_and_create_tmp_path(self):
if not os.path.isdir(self.tmp_path):
os.mkdir(self.tmp_path)



@beartype.beartype
def __retrieve_tif(self, filename: str):
"""
Expand All @@ -495,29 +498,37 @@ def __retrieve_tif(self, filename: str):
_type_: The open geotiff in a gdal handler
"""
self.__check_and_create_tmp_path()

# For gdal debugging use exceptions
gdal.UseExceptions()
bb_ll = tuple(self.bounding_box_polygon.to_crs("EPSG:4326").geometry.total_bounds)
# try:

if filename.lower() == "aus" or filename.lower() == "au":

url = "http://services.ga.gov.au/gis/services/DEM_SRTM_1Second_over_Bathymetry_Topography/MapServer/WCSServer?"
wcs = WebCoverageService(url, version="1.0.0")

coverage = wcs.getCoverage(
identifier="1", bbox=bb_ll, format="GeoTIFF", crs=4326, width=2048, height=2048
)
# This is stupid that gdal cannot read a byte stream and has to have a
# file on the local system to open or otherwise create a gdal file
# from scratch with Create

tmp_file = os.path.join(self.tmp_path, "StupidGDALLocalFile.tif")

with open(tmp_file, "wb") as fh:
fh.write(coverage.read())
tif = gdal.Open(tmp_file)

elif filename == "hawaii":
import netCDF4

bbox_str = (
f"[({str(bb_ll[1])}):1:({str(bb_ll[3])})][({str(bb_ll[0])}):1:({str(bb_ll[2])})]"
)


filename = f"https://pae-paha.pacioos.hawaii.edu/erddap/griddap/srtm30plus_v11_land.nc?elev{bbox_str}"
f = urllib.request.urlopen(filename)
ds = netCDF4.Dataset("in-mem-file", mode="r", memory=f.read())
Expand Down Expand Up @@ -723,7 +734,10 @@ def parse_structure_map(self) -> tuple:
structure["DIPDIR"] = self.raw_data[Datatype.STRUCTURE][config["dipdir_column"]]
else:
print(f"Structure map does not contain dipdir_column '{config['dipdir_column']}'")


# Ensure all DIPDIR values are within [0, 360]
structure["DIPDIR"] = structure["DIPDIR"] % 360.0

if config["dip_column"] in self.raw_data[Datatype.STRUCTURE]:
structure["DIP"] = self.raw_data[Datatype.STRUCTURE][config["dip_column"]]
else:
Expand Down
Loading
Loading