Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General CSV reader #152

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

General CSV reader #152

wants to merge 5 commits into from

Conversation

hugary1995
Copy link
Collaborator

@hugary1995 hugary1995 commented Jun 23, 2024

This PR adds a header-only CSV reader from https://github.com/vincentlaucsb/csv-parser. I didn't include the library as a cmake subdirectory because it's folder hierarchy makes install very difficult. Since it is MIT license, I just copied the single header file into our contrib folder.

The primary entry point for CSV reader is Sequence. It is designed as a sequence reader to retrieve a single column from a CSV file. Design details are included in the class docstring. It's zero-effort for developers to opt-in to use a CSV to fill a vector by changing

options.set<std::vector<Real>>("foo");

to

options.set<Sequence<Real>>("foo");

With the above code, users are able to specify the sequence in one of three ways:

  1. foo = '1 2 3 4 5 6' <-- equivalent to parsing std::vector<Real>
  2. foo = 'my.csv:key' <-- we try to read my.csv and fill the vector with the CSV column with name key.
  3. foo = 'my.csv:[2]' <-- we try to read my.csv and fill the vector with the CSV column with index 2.

Other changes are just for expanding parser support and test coverage.

This is the (big) first step towards @sapitts 's request of defining irradiation eigenstrain using a CSV file.

close #154

Copy link

github-actions bot commented Jun 23, 2024

PR Preview Action v1.4.7
🚀 Deployed preview to https://reverendbedford.github.io/neml2/pr-preview/pr-152/
on branch gh-pages at 2024-06-23 02:25 UTC

@reverendbedford
Copy link
Owner

Syntax check

No syntax error, good job! 💜

Copy link

github-actions bot commented Jun 23, 2024

Python Binding Test Results (macos-12)

63 tests  ±0   39 ✅ ±0   2s ⏱️ -1s
 1 suites ±0   24 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

♻️ This comment has been updated with latest results.

Copy link

Test Results (macos-12-Release-ON)

    3 files  ± 0      3 suites  ±0   1m 4s ⏱️ -3s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 079 runs  +14  3 079 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Test Results (macos-12-Debug-ON)

    3 files  ± 0      3 suites  ±0   2m 34s ⏱️ +18s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 081 runs  +14  3 081 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Test Results (ubuntu-latest-Release-ON)

    3 files  ± 0      3 suites  ±0   52s ⏱️ -1s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 079 runs  +14  3 079 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Test Results (ubuntu-latest-Debug-ON)

    3 files  ± 0      3 suites  ±0   2m 1s ⏱️ -6s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 081 runs  +14  3 081 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Python Binding Test Results (ubuntu-latest)

63 tests  ±0   39 ✅ ±0   2s ⏱️ ±0s
 1 suites ±0   24 💤 ±0 
 1 files   ±0    0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Test Results (macos-12-Release-OFF)

    3 files  ± 0      3 suites  ±0   1m 10s ⏱️ -17s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 079 runs  +14  3 079 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Test Results (macos-12-Debug-OFF)

    3 files  ± 0      3 suites  ±0   2m 20s ⏱️ +3s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 081 runs  +14  3 081 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Test Results (ubuntu-latest-Release-OFF)

    3 files  ± 0      3 suites  ±0   51s ⏱️ ±0s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 079 runs  +14  3 079 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

Copy link

Test Results (ubuntu-latest-Debug-OFF)

    3 files  ± 0      3 suites  ±0   2m 4s ⏱️ ±0s
  495 tests + 5    495 ✅ + 5  0 💤 ±0  0 ❌ ±0 
3 081 runs  +14  3 081 ✅ +14  0 💤 ±0  0 ❌ ±0 

Results for commit 84a866a. ± Comparison against base commit 067f935.

@reverendbedford
Copy link
Owner

Coverage after merging csv_reader into main will be

86.91%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
include/neml2/base
   CrossRef.h34.21%100%31.37%58.33%105, 107–108, 72, 79
   DependencyResolver.h86.13%100%88.24%85.83%219, 221–222, 226, 229–231, 264, 291, 303, 313–314, 344, 346–347, 349–350
   Factory.h39.42%100%24.69%91.30%148, 156
   NEML2Object.h45.45%100%20%66.67%50, 52, 63, 66
   OptionCollection.h0%100%0%0%40, 49
   OptionSet.h32.93%100%30.02%57.69%128, 134, 137, 143, 149, 214, 216–217, 379, 381–383, 385, 61, 66, 79, 81, 85, 89, 91, 93, 95
   Registry.h80.04%100%79.81%100%
   Sequence.h6.06%100%3.33%10.26%115, 118–119, 123–124, 126–130, 136–137, 141, 144–146, 148, 152, 156–157, 159, 162–165, 167–168, 171–172, 176, 178–179, 64, 80, 83
   Storage.h56.10%100%39.13%77.78%147–148, 150, 177
include/neml2/drivers
   Driver.h0%100%0%0%56
   TransientDriver.h85.71%100%100%75%149
include/neml2/misc
   error.h30.81%100%28.88%63.64%37–38, 40, 94, 96–99
   math.h100%100%100%100%
   parser_utils.h42.96%100%32.73%88%37–38, 40
   utils.h56.40%100%47.30%77.78%173, 239, 241–246, 267, 302, 304–307
include/neml2/models
   BufferStore.h34.43%100%18.92%58.33%113, 115, 117–121, 133, 147, 151
   ComposedModel.h100%100%100%100%
   Data.h100%100%100%100%
   Interpolation.h69.81%100%55.56%100%
   LinearInterpolation.h20%100%8.57%100%
   Model.h85.19%100%75%89.47%116, 75
   NonlinearParameter.h10.53%100%5.56%100%
   ParameterStore.h21.43%100%5.41%52.63%132, 134, 136–140, 148, 152
   VariableStore.h62.45%100%54.59%89.47%236, 238–239, 260, 91, 97
include/neml2/models/crystallography
   CrystalGeometry.h0%100%0%0%90, 92
   MillerIndex.h100%100%100%100%
   crystallography.h100%100%100%100%
include/neml2/solvers
   Newton.h100%100%100%100%
   NonlinearSystem.h66.67%100%66.67%66.67%66
include/neml2/tensors
   BatchTensorBase.h59.80%100%44.68%93.65%212, 214, 256, 258
   FixedDimTensor.h26.44%100%19.32%65.63%152, 156, 168, 171, 176, 178, 183, 188, 66, 89, 91
   LabeledAxis.h0%100%0%0%142, 67, 72, 74–75, 77
   LabeledAxisAccessor.h72.50%100%65.38%85.71%46, 77
   LabeledTensor.h56.04%100%46.88%77.78%108, 112, 127, 176, 178–179
   Scalar.h44%100%34.55%70%82, 84–88
   TensorValue.h14%100%10.87%50%39, 63, 65, 75
   Transformable.h0%100%0%0%44
   Variable.h40%100%31.06%82%101, 107, 43–47, 49, 51
   VecBase.h80.95%100%50%100%
src/neml2/base
   CrossRef.cxx83.87%100%75%100%
   Factory.cxx100%100%100%100%
   HITParser.cxx100%100%100%100%
   NEML2Object.cxx91.67%100%100%90%35
   OptionCollection.cxx100%100%100%100%
   OptionSet.cxx97.67%100%100%96.88%35
   Parser.cxx66.67%100%100%64.29%39–42, 54
   Registry.cxx75%100%66.67%76.92%38, 40–41, 56, 58–59
src/neml2/drivers
   Driver.cxx91.67%100%100%90%39
   TransientDriver.cxx86.60%100%100%85.39%142, 239–240, 245–247, 250–251, 255–259, 261–266, 269, 274–278, 99
src/neml2/drivers/solid_mechanics
   LargeDeformationIncrementalSolidMechanicsDriver.cxx32.39%100%25%32.84%100–102, 104–105, 109, 111, 113, 117, 119, 121, 125, 127–128, 130–131, 133–134, 137, 139–143, 63, 65–66, 68–74, 76, 78–79, 81, 83–84, 91–92, 94–95, 98
   SolidMechanicsDriver.cxx73.58%100%100%72.55%104, 106–107, 117, 148, 150–151, 154, 156, 158, 161, 163, 168–169, 171–173, 175, 177–178, 180, 194–196, 200–202, 77
src/neml2/misc
   error.cxx100%100%100%100%
   math.cxx99.44%100%100%99.33%252
   parser_utils.cxx98.46%100%100%98.18%73
   types.cxx100%100%100%100%
   utils.cxx89.47%100%100%86.67%44, 52
src/neml2/models
   ArrheniusParameter.cxx97.14%100%100%96.88%53
   BackwardEulerTimeIntegration.cxx94.12%100%77.78%97.62%56
   BufferStore.cxx33.33%100%33.33%33.33%36, 38, 40, 44, 46, 48–50
   ComposedModel.cxx95.31%100%100%94.96%149, 151, 195, 56, 86, 92
   ConstantParameter.cxx52.11%100%40.74%88.24%41, 59
   CopyVariable.cxx35.62%100%33.33%42.11%46, 49, 51–52, 54, 58, 60–61, 63–64, 70
   Data.cxx90%100%100%87.50%35
   ForceRate.cxx98%100%100%97.73%49
   ForwardEulerTimeIntegration.cxx97.67%100%100%97.30%51
   ImplicitUpdate.cxx98.39%100%100%98.28%47
   LinearInterpolation.cxx61.18%100%40.74%96.77%42
   Model.cxx91.25%100%94.44%90.80%141–144, 177–179, 286, 288–291, 297, 422, 424–426, 428–429, 51, 79–80, 88, 96
   NonlinearParameter.cxx61.90%100%55.56%100%
   ParameterStore.cxx48.08%100%22.73%66.67%101, 47, 49, 51–53, 66, 93, 95–96
   RotationMatrix.cxx95.83%100%100%95.24%45
   SR2Invariant.cxx97.14%100%100%97.01%131, 48
   StateRate.cxx94%100%66.67%97.73%49
   SumModel.cxx86.96%100%100%85%75–76, 79, 81–83
   VariableStore.cxx94.25%100%91.67%94.67%165–166, 63, 65
   WR2ExplicitExponentialTimeIntegration.cxx97.37%100%100%97.14%47
   WR2ImplicitExponentialTimeIntegration.cxx97.56%100%100%97.37%54
src/neml2/models/crystallography
   CrystalGeometry.cxx98.46%100%100%98.25%198, 61
   CubicCrystal.cxx93.75%100%100%92.86%51
   MillerIndex.cxx100%100%100%100%
   crystallography.cxx98.28%100%100%98.08%142
src/neml2/models/crystallography/user_tensors
   FillMillerIndex.cxx94.44%100%100%93.33%44
   SymmetryFromOrbifold.cxx93.33%100%100%92.31%47
src/neml2/models/solid_mechanics
   AssociativeIsotropicPlasticHardening.cxx96.97%100%100%96.67%51
   AssociativeKinematicPlasticHardening.cxx97.06%100%100%96.77%53

Copy link
Owner

@reverendbedford reverendbedford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use this to parse the vtest files as well?

@hugary1995
Copy link
Collaborator Author

hugary1995 commented Jun 25, 2024

I just merged #151 so I'll have to rebase this one.

Can we use this to parse the vtest files as well?

Mechanistically yes. Though right now I rely on the glob pattern *.csv:* to distinguish between std::vector<std::string> and a CSV file column. This requires the file extension to be .csv. We need to think of some other qualification for vtest. For example, it'll be kinda neat if I can detect both .csv AND .vtest and automatically choose a parsing format -- I'll give it a shot after rebasing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use CSV file to define forces
2 participants