-
Notifications
You must be signed in to change notification settings - Fork 518
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Engineering Design Interface (EDI) Contrib Package #2937
base: main
Are you sure you want to change the base?
Changes from all commits
b354844
3377110
604b0a1
eed941f
f5a3232
d2696b6
eba2990
f625ef2
b8dd90e
4603e40
39ea960
b818da7
fe2821f
550fb70
f3d306f
ab02aba
4e3f1d1
d62b011
39a5211
09064b4
df6336b
b369e68
e594986
3a7918b
a1049a8
816a022
871014c
6c4ef32
a6cde71
f76f497
7542063
b75d75f
e5fbc0e
10d6845
6df90fd
cc683cb
2acc63e
fd0c45b
7a58723
e769553
4ebe52a
48abb03
ca94021
085e729
36b99f9
4381d3b
98c3ef6
7635017
2d4a0ed
2c48406
29dd11b
871c8ed
13c58af
ba8c1f3
ae6c1be
f186d89
7acdac4
f18548d
1237116
782696f
13bc460
e07dce3
1850267
54d50eb
7def60b
28df353
e255302
65f23dd
d3b8dcd
ef135d1
1fb0c8d
3a2ea94
ac487b8
808483f
2342baa
42d2357
b101ab4
0ef183f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
Additional Tips | ||
--------------- | ||
|
||
* Developers may need to install the following additional packages: | ||
|
||
:: | ||
|
||
pip install pytest | ||
pip install pytest-cov | ||
pip install sphinx | ||
pip install sphinx_rtd_theme | ||
pip install sphinx_copybutton | ||
|
||
|
||
* If you wish to build the documentation locally, use: | ||
|
||
:: | ||
|
||
cd <path_to_pyomo>/pyomo/doc/OnlineDocs | ||
make html | ||
|
||
then open the file ``<path_to_pyomo/doc/OnlineDocs/_build/html/index.html>`` | ||
|
||
|
||
* Unit tests and coverage can be run locally using: | ||
|
||
:: | ||
|
||
cd <path_to_pyomo>/pyomo/pyomo/contrib/edi | ||
pytest --cov-report term-missing --cov=pyomo.contrib.edi -v ./tests/ | ||
|
||
or generating html output: | ||
|
||
:: | ||
|
||
cd <path_to_pyomo>/pyomo/pyomo/contrib/edi | ||
pytest --cov-report html --cov=pyomo.contrib.edi -v ./tests/ | ||
Comment on lines
+25
to
+37
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Similar to my previous comment, maybe it would be more beneficial to add a section to the contribution guide on how to run tests for a specific subset of the repo. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with this - and it shows a miss in our documentation. We should table this for after the doc-reorg. |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,115 @@ | ||||||
Advanced Runtime Constraints | ||||||
============================ | ||||||
|
||||||
|
||||||
Automated unpacking for multi-use black-boxes | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should have a design discussion about the functionality described in this section. There has to be a better way to support both scalar and vectorized operation. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file has been removed from the doc tree |
||||||
--------------------------------------------- | ||||||
|
||||||
Coding a black-box model often represents a significant effort, and it is therefore desirable to have the black-box work in many situations. A common case is to have a black-box model with a scalar input variable perform vectorized operations, ie, take in a vector and return a vector. In the more general case, this can be thought of as passing in multiple run-cases as input to the black-box model. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file has been removed from the documentation |
||||||
|
||||||
The ``parseInputs()`` method enables this batch-like capability in a variety of forms: | ||||||
|
||||||
.. py:function:: BlackBoxFunctionModel.parseInputs(self, *args, **kwargs) | ||||||
|
||||||
Parses the inputs to a black-box model into a list of run-cases | ||||||
|
||||||
:param args: The self attribute in all python methods | ||||||
:type self: black-box model | ||||||
:param args: index passed arguments | ||||||
Comment on lines
+16
to
+18
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this file has been removed from the doc tree |
||||||
:type args: list or tuple | ||||||
:param kwargs: keyword passed arguments | ||||||
:type kwargs: dict | ||||||
|
||||||
:return: runcases | ||||||
:rtype: list | ||||||
:return: returnMode | ||||||
:rtype: int | ||||||
:return: extras | ||||||
:rtype: list | ||||||
Comment on lines
+12
to
+28
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Make this a docstring instead. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. addressed above |
||||||
|
||||||
|
||||||
The function definition is not particularly helpful, so let's dive in a bit. For the typical user, we recommend that the top of all ``BlackBox()`` methods appear as follows: | ||||||
|
||||||
.. literalinclude:: ../../../../pyomo/contrib/edi/tests/test_docSnippets.py | ||||||
:language: python | ||||||
:dedent: 12 | ||||||
:start-after: # BEGIN: AdvancedRTC_Snippet_02 | ||||||
:end-before: # END: AdvancedRTC_Snippet_02 | ||||||
|
||||||
|
||||||
Essentially, ``parseInputs()`` is a pre-processor that directly takes the inputs of the black-box. The ``parseInputs()`` method will check all of the inputs, ensure that size and units are correct, split into run cases as appropriate, and return a run-cases list that is ready to operate on. | ||||||
|
||||||
The ``runCases`` return (which can be named as any valid python name) is a list of dicts, where the keys to the dict are the names of the inputs declared in the ``__init__()`` method. Ex: ``runCases[0]['x']`` will give the ``'x'`` input (in units specified in the ``__init__()`` method) in the first run-case. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file has been removed from the doc tree |
||||||
|
||||||
The ``returnMode`` return (which can be named as any valid python name) is an integer that indicates how the return should be processed. If ``returnMode`` is passed as a ``kwarg``, then the passed in value will be cast to this output. If it is not passed in, then ``returnMode`` will be either ``-1*self.availableDerivative``, indicating that there is only a single run case, or ``self.availableDerivative`` which indicated multiple run cases. A negative value for ``returnMode`` indicates that there will be one less layer of indexing in the output. | ||||||
|
||||||
The ``extras`` return (which can be named as any valid python name) is a dict that will include all of the additional passed in values not expected by the black-box model. The extra ``kwargs`` will appear as normal, and extra ``args`` get put in a list in ``extras['remainingArgs']``. If you pass in a ``kwarg`` with key name ``'remainingArgs'``, it will be overwritten. The extras dict is the place to find options that may affect the code (ex: tolerances, run modes, etc) that are not expected inputs to the black-box model. | ||||||
|
||||||
Note that when using run case input, the output will always take the following unpacking structure: | ||||||
|
||||||
:: | ||||||
|
||||||
output[<run_case_index>][0] = <list_of_outputs_for_specified_runcase> | ||||||
output[<run_case_index>][0][<index_of_output>] = <output_for_specified_runcase> | ||||||
|
||||||
output[<run_case_index>][1] = <list_of_jacobians_for_specified_runcase> | ||||||
output[<run_case_index>][1][<index_of_output>][<index_of_input>] = <d(output_of_specified_index)/d(input_of_specified_index)> | ||||||
|
||||||
|
||||||
There is **not** a shortcut for single scalar output black boxes as is the case when not using run cases, the final index to output 0 must be included. | ||||||
|
||||||
|
||||||
|
||||||
An Example | ||||||
++++++++++ | ||||||
|
||||||
There are many ways this functionality can be used, we provide an example here to get new users started | ||||||
|
||||||
.. literalinclude:: ../../../../pyomo/contrib/edi/tests/test_docSnippets.py | ||||||
:language: python | ||||||
:dedent: 8 | ||||||
:start-after: # BEGIN: AdvancedRTC_Snippet_01 | ||||||
:end-before: # END: AdvancedRTC_Snippet_01 | ||||||
|
||||||
|
||||||
Check outputs | ||||||
------------- | ||||||
|
||||||
There is a ``checkOutputs()`` method that is not supported in the current version. Contact the developers if you desire this functionality, but the following the practices described in this documentation should render the need for this moot. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This file has been removed from the doc tree |
||||||
|
||||||
|
||||||
Cases of non-scalar inputs or outputs | ||||||
------------------------------------- | ||||||
|
||||||
Indexing can get somewhat complicated when inputs and outputs are not scalars. Users should be warned this feature is supported, but not well tested, so please contact the developers if you encounter any unusual behavior. | ||||||
|
||||||
The following unpacking remains the same: | ||||||
|
||||||
:: | ||||||
|
||||||
output[0] = <list_of_outputs> | ||||||
output[0][<index_of_output>] = <output> | ||||||
|
||||||
output[1] = <list_of_jacobians> | ||||||
output[1][<index_of_output>][<index_of_input>] = <d(output_of_specified_index)/d(input_of_specified_index)> | ||||||
|
||||||
However, for outputs, the result will be an array with dimensions equal to the size of the output. For Jacobians, it breaks down as the following: | ||||||
|
||||||
:: | ||||||
|
||||||
jacobian_list_entry[(output_dim_1_ix, output_dim_2_ix, ..., input_dim_1_ix, input_dim_2_ix, ...)] = <scalar_d(output_of_specified_index)/d(input_of_specified_index)> | ||||||
|
||||||
For example, with an output that is 2x2 and an input that is also 2x2 | ||||||
|
||||||
:: | ||||||
|
||||||
output[1][<index_of_output>][<index_of_input>][(0,0,1,1)] | ||||||
|
||||||
is the derivative of ``output[0,0]`` with respect to ``input[1,1]`` | ||||||
|
||||||
|
||||||
|
||||||
Tips | ||||||
---- | ||||||
|
||||||
* A model summary can be printed by calling ``print(model_instance.summary)`` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of this seems generic and not specific to EDI. Should this get moved to the "contribution" section of Pyomo's documentation?
There are also some build options that can be specified when
pip
installing Pyomo that will grab the packages needed to run tests or build documentation. Maybe we should point the users to those rather than manually listing packages? I'm mainly concerned that this list could become out of sync with the requirements specified insetup.py
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
setup.py
does not includepytest-cov
https://github.com/Pyomo/pyomo/blob/main/setup.py#L232
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Current documentation only lists:
but it is not clear if a developer would also have to run the following