config: point `pa2024ch` to data prep with new weo 2023 scenarios #333

jdhoffa · 2024-07-01T14:20:23Z

The newly created data directory now contains APS and STEPS scenarios for WEO 2023 for the aviation, cement and steel sectors.

Importantly also, we had to update paths to the new (real) peer files, as those had to be re-run with those same latest scenarios.

With these scenarios missing, the executive summary couldn't render correctly.

Relates to:
RMI-PACTA/workflow.data.preparation/pull/238
RMI-PACTA/workflow.data.preparation/pull/237
RMI-PACTA/workflow.data.preparation/pull/236
RMI-PACTA/pacta.scenario.data.preparation/pull/61
RMI-PACTA/pacta.scenario.data.preparation/pull/63

The newly created data directory now contains APS and STEPS scenarios for WEO 2023 for the aviation, cement and steel sectors. With these scenarios missing, the executive summary couldn't render correctly. Relates to: RMI-PACTA/workflow.data.preparation/pull/238 RMI-PACTA/workflow.data.preparation/pull/237 RMI-PACTA/workflow.data.preparation/pull/236 RMI-PACTA/pacta.scenario.data.preparation/pull/61 RMI-PACTA/pacta.scenario.data.preparation/pull/63

jdhoffa · 2024-07-01T14:21:17Z

Open question to @AlexAxthelm and @cjyetman do you think I should update the GENERAL config as well? I don't think it is strictly necessary to do so... BUT I also can't think fo a strong reason NOT to.

It might make life less confusing down the road to keep both initiatives with the same data version.

cjyetman · 2024-07-01T14:35:38Z

I think that we have said that whatever happens with WEO will happen everywhere (PA2024CH, GENERAL, and all 2023Q4 initiatives), and I expect that word has traveled around, so I think we should follow through on that.

Additionally, until this issue is resolved, I prefer that we don't get into using different 2023Q4 datasets for different initiatives:
RMI-PACTA/workflow.data.preparation#226

jdhoffa · 2024-07-01T14:39:40Z

Cool! done

jdhoffa · 2024-07-01T14:41:47Z

Going to wait for checks to pass and inspect the outputs prior to merging

cjyetman · 2024-07-02T10:48:05Z

presumably if #331 merges and this PR branch is updated, the tests will pass? so we're waiting on that, no?

jdhoffa · 2024-07-02T10:50:03Z

Indeed, depends on #331

I will re-trigger CI/CD once that is merged.

github-actions · 2024-07-02T13:19:47Z

Docker build status

commit_time	git_sha	project_code	holdings_date	language	peer_group	download_command	image
2024-07-17T09:30:50Z	`0c40429`,	GENERAL	2022Q4	EN	other	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports/rmi_pacta_2022q4_general-20240717T142911Z/EN/other/2/working_dir/50_Outputs/rmi_pacta_2022q4_general --destination rmi_pacta_2022q4_general-EN-other`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2022q4_general:20240717T142911Z`
2024-07-17T09:30:50Z	`0c40429`,	GENERAL	2023Q4	EN	other	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports/rmi_pacta_2023q4_general-20240717T142911Z/EN/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_general --destination rmi_pacta_2023q4_general-EN-other`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_general:20240717T142911Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	DE	bank	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/DE/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-DE-bank`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	DE	other	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/DE/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-DE-other`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	EN	assetmanager	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/assetmanager/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-assetmanager`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	EN	bank	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-bank`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	EN	insurance	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/insurance/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-insurance`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	EN	other	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-other`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	EN	pensionfund	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/pensionfund/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-pensionfund`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	FR	bank	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/FR/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-FR-bank`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`,	PA2024CH	2023Q4	FR	other	`az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/FR/other/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-FR-other`	`transitionmonitordockerregistry.azurecr.io/rmi_pacta_2023q4_pa2024ch:20240717T142912Z`
2024-07-17T09:30:50Z	`0c40429`						ghcr.io/rmi-pacta/workflow.transition.monitor:pr333

cjyetman · 2024-07-02T13:22:59Z

🤔
https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports/rmi_pacta_2023q4_pa2024ch-20240702T122952Z/EN/assetmanager/1/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch/executive_summary/template.pdf

jdhoffa · 2024-07-02T13:37:56Z

shit!

jdhoffa · 2024-07-02T13:46:06Z

Logs for this run can be found here:
https://portal.azure.com/#view/Microsoft_Azure_Storage/ContainerMenuBlade/~/overview/storageAccountId/%2Fsubscriptions%2Ffeef729b-4584-44af-a0f9-4827075512f9%2FresourceGroups%2FRMI-SP-PACTA-DEV%2Fproviders%2FMicrosoft.Storage%2FstorageAccounts%2Fpactadatadev/path/ghactions-workflow-transition-monitor-results-full/etag/%220x8DC3D63D9990D41%22/defaultEncryptionScope/%24account-encryption-key/denyEncryptionScopeOverride~/false/defaultId//publicAccessVal/None

jdhoffa · 2024-07-02T13:46:24Z

error_messages.txt was helpful XD

2024-07-02 13:16:23.402782 ES: There was an error in prep/plot_scores(). Returning empty plot object.

2024-07-02 13:16:23.616256 ES: There was an error in prep/plot_scores(). Returning empty plot object.

Which is actually super bizarre since that isn't even the expected error message...
https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L129-L146

This might be a cache invalidation thing.

AlexAxthelm · 2024-07-02T14:35:29Z

error_messages.txt was helpful XD
2024-07-02 13:16:23.402782 ES: There was an error in prep/plot_scores(). Returning empty plot object.

2024-07-02 13:16:23.616256 ES: There was an error in prep/plot_scores(). Returning empty plot object.
Which is actually super bizarre since that isn't even the expected error message... https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L129-L146

This might be a cache invalidation thing.

I think you might be looking at the wrong lines. The message you're getting is at line 354 & 377

jdhoffa · 2024-07-02T16:42:32Z

https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L354

and

https://github.com/RMI-PACTA/pacta.executive.summary/blob/14575e1433e229154b84049d9a6b9c406e9d8366/inst/extdata/PA2024CH_en_exec_summary/scorecard.Rmd#L377

don't have anything interesting on them? Are we talking about different files?

jdhoffa · 2024-07-03T11:25:49Z

@MonikaFu FYI, the problem seems to be that some of the sectors yield NA scores
When I run:

data_scores_b <- prep_scores(
      results_portfolio = results_portfolio,
      peers_results_aggregated = peers_results_aggregated,
      asset_class = "bonds",
      scenario_source = scenario_source
    )

I get:

or

# A tibble: 14 × 5
   asset_class scope     entity         sector     score
   <chr>       <chr>     <chr>          <chr>      <chr>
 1 bonds       portfolio peers          NA         E    
 2 bonds       sector    peers          automotive C    
 3 bonds       sector    peers          aviation   NA   
 4 bonds       sector    peers          cement     NA   
 5 bonds       sector    peers          coal       E    
 6 bonds       sector    peers          gas        E    
 7 bonds       sector    peers          oil        E    
 8 bonds       sector    peers          power      E    
 9 bonds       sector    peers          steel      NA   
10 bonds       portfolio this_portfolio NA         A+   
11 bonds       sector    this_portfolio gas        A    
12 bonds       sector    this_portfolio oil        C    
13 bonds       sector    this_portfolio power      A+   
14 bonds       sector    this_portfolio steel      E

This seems to be what is causing the plot error.

jdhoffa · 2024-07-03T11:33:05Z

This is the error I get:

Error in `abort_if_invalid_values()` at pacta.executive.summary/R/plot_scores.R:227:3:
! Each value of `"sector"` must be one of these:
NA, power, automotive, coal, oil, gas, aviation, steel.
✖ You passed: cement.
Run `rlang::last_trace()` to see where the error occurred.

@MonikaFu

@MonikaFu unless there is a specific reason not to? Relates to RMI-PACTA/workflow.transition.monitor#333 and in particular RMI-PACTA/workflow.transition.monitor#333 (comment)

jdhoffa · 2024-07-03T11:35:47Z

Probably solved by RMI-PACTA/pacta.executive.summary#348

@MonikaFu

* bug: allow `cement` as an input sector to `plot_scores` @MonikaFu unless there is a specific reason not to? Relates to RMI-PACTA/workflow.transition.monitor#333 and in particular RMI-PACTA/workflow.transition.monitor#333 (comment) * bump snaps

jdhoffa · 2024-07-04T08:49:10Z

@AlexAxthelm @cjyetman @MonikaFu
After some sleuthing locally, I have determined that the scorecard data still won't generate since the peer data does NOT contain the updated scenarios for the emission sectors.

What that means we need to re-run the peer data (might already have been done by @AlexAxthelm , need to make sure the latest run contains STEPS, APS and NZE for the aviation, cement and steel sectors) and then either:

Update the config to point to that new data OR
Just update the existing testing peer data

This runs in parallel to a discussion we've already been having about updating our peer data in the CI/CD pipeline.

Converting this PR to draft until the above has been completed.

jdhoffa · 2024-07-09T16:18:53Z

Latest bump of data_share_path now incorporates changes from RMI-PACTA/pacta.data.preparation#32

jdhoffa · 2024-07-15T11:23:35Z

For the record: after a call with @AlexAxthelm we realized that we need the appropriate versions of peer_results and user_data also configured appropriately in this PR.

To that end, I have asked him to target #333 with the #340 PR, and will merge it that together with this PR (I will update the PR title and description appropriately).

I will also wait until @MonikaFu runs and updates the user_data (and get that on Azure` before re-running CI/CD here.

After all the above is complete, we should have a good idea of if all IR and ES building runs well "in the wild".
I would expect all plots to render correctly.

cjyetman · 2024-07-18T07:21:25Z

build/config/rmi_pacta_2023q4_general.json

@@ -1,5 +1,5 @@
 {
-  "data_share_path": "2023Q4_20240424T120055Z",
+  "data_share_path": "2023Q4_20240709T111731Z",


side note: data_share_path is not a very intuitive name for the PACTA input directory

cjyetman · 2024-07-18T07:24:12Z

build/config/rmi_pacta_2023q4_pa2024ch.json

  "pacta_data_quarter": "2023Q4",
  "project_code": "PA2024CH",
  "templates_ref": "",
-  "peer_results": "https://pactadatadev.blob.core.windows.net/testing-files/peer-results/PA2024CH_peer-results-test",
+  "peer_results": "https://pactadatadev.blob.core.windows.net/project-files/pa2024ch/peer_files",


I know it's mentioned in the (very long) thread of this PR, but it's not super clear to me why/if the peer files should change here.

Indeed!
So we are shifting to the real peer-results file (as opposed to the test files previously) since they had to be re-run to correctly contain the appropriate scenario data, as per RMI-PACTA/pacta.scenario.data.preparation#61

(This is also why the testing matrix has changed for CI/CD, since we are using the real files, the URL has been replaced with an az storage copy command)

Would you like me to document that in the PR body?

cjyetman

I can't / don't know how to review the test results anymore, but otherwise seems reasonable

jdhoffa · 2024-07-18T07:57:56Z

I can't / don't know how to review the test results anymore, but otherwise seems reasonable

To test you can run this command:

az storage copy --recursive --source https://pactadatadev.blob.core.windows.net/ghactions-workflow-transition-monitor-results-reports-private/rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-bank

However, since I believe you were never able to get the az cli up and running? It is also possible to access the files from the Azure GUI.

From portal.azure.com, navigate to the pactadatadev > Storage Browser > Blob containers > ghactions-workflow-transition-monitor-results-reports-private and then find the file path as usual rmi_pacta_2023q4_pa2024ch-20240717T142912Z/EN/bank/2/working_dir/50_Outputs/rmi_pacta_2023q4_pa2024ch --destination rmi_pacta_2023q4_pa2024ch-EN-bank

I know it's not the most intuitive but definitely possible if you have the gumption 😉

bump general to reference new data path

8ee300f

Merge branch 'main' into jdhoffa-patch-1

651fe1c

jdhoffa marked this pull request as ready for review July 1, 2024 14:40

jdhoffa requested a review from cjyetman as a code owner July 1, 2024 14:40

cjyetman previously approved these changes Jul 1, 2024

View reviewed changes

cjyetman self-requested a review July 2, 2024 10:44

jdhoffa mentioned this pull request Jul 2, 2024

Add a parameter to executive summary calculations #331

Merged

Merge branch 'main' into jdhoffa-patch-1

90d1449

jdhoffa mentioned this pull request Jul 3, 2024

bug: allow cement as an input sector to plot_scores RMI-PACTA/pacta.executive.summary#348

Merged

jdhoffa dismissed cjyetman’s stale review via 34d7ccd July 3, 2024 13:34

revert this: point to PR branch for testing

34d7ccd

jdhoffa marked this pull request as draft July 4, 2024 08:44

revert: dont point to PR branch

825751e

jdhoffa added 2 commits July 9, 2024 18:19

bump to latest run of data prep

e25c491

Merge branch 'main' into jdhoffa-patch-1

54f0086

jdhoffa mentioned this pull request Jul 15, 2024

Build/336 update peer data #340

Merged

Build/336 update peer data (#340)

0c40429

jdhoffa marked this pull request as ready for review July 18, 2024 06:56

cjyetman reviewed Jul 18, 2024

View reviewed changes

cjyetman approved these changes Jul 18, 2024

View reviewed changes

jdhoffa merged commit 081a997 into main Jul 18, 2024
22 checks passed

jdhoffa deleted the jdhoffa-patch-1 branch July 18, 2024 08:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

config: point `pa2024ch` to data prep with new weo 2023 scenarios #333

config: point `pa2024ch` to data prep with new weo 2023 scenarios #333

jdhoffa commented Jul 1, 2024 •

edited

Loading

jdhoffa commented Jul 1, 2024

cjyetman commented Jul 1, 2024

jdhoffa commented Jul 1, 2024

jdhoffa commented Jul 1, 2024

cjyetman commented Jul 2, 2024

jdhoffa commented Jul 2, 2024

github-actions bot commented Jul 2, 2024 •

edited

Loading

cjyetman commented Jul 2, 2024

jdhoffa commented Jul 2, 2024

jdhoffa commented Jul 2, 2024

jdhoffa commented Jul 2, 2024 •

edited

Loading

AlexAxthelm commented Jul 2, 2024

jdhoffa commented Jul 2, 2024 •

edited

Loading

jdhoffa commented Jul 3, 2024 •

edited

Loading

jdhoffa commented Jul 3, 2024

jdhoffa commented Jul 3, 2024

jdhoffa commented Jul 4, 2024

jdhoffa commented Jul 9, 2024

jdhoffa commented Jul 15, 2024

cjyetman Jul 18, 2024

cjyetman Jul 18, 2024

jdhoffa Jul 18, 2024

jdhoffa Jul 18, 2024

cjyetman left a comment

jdhoffa commented Jul 18, 2024 •

edited

Loading

config: point pa2024ch to data prep with new weo 2023 scenarios #333

config: point pa2024ch to data prep with new weo 2023 scenarios #333

Conversation

jdhoffa commented Jul 1, 2024 • edited Loading

jdhoffa commented Jul 1, 2024

cjyetman commented Jul 1, 2024

jdhoffa commented Jul 1, 2024

jdhoffa commented Jul 1, 2024

cjyetman commented Jul 2, 2024

jdhoffa commented Jul 2, 2024

github-actions bot commented Jul 2, 2024 • edited Loading

Docker build status

cjyetman commented Jul 2, 2024

jdhoffa commented Jul 2, 2024

jdhoffa commented Jul 2, 2024

jdhoffa commented Jul 2, 2024 • edited Loading

AlexAxthelm commented Jul 2, 2024

jdhoffa commented Jul 2, 2024 • edited Loading

jdhoffa commented Jul 3, 2024 • edited Loading

jdhoffa commented Jul 3, 2024

jdhoffa commented Jul 3, 2024

jdhoffa commented Jul 4, 2024

jdhoffa commented Jul 9, 2024

jdhoffa commented Jul 15, 2024

cjyetman Jul 18, 2024

Choose a reason for hiding this comment

cjyetman Jul 18, 2024

Choose a reason for hiding this comment

jdhoffa Jul 18, 2024

Choose a reason for hiding this comment

jdhoffa Jul 18, 2024

Choose a reason for hiding this comment

cjyetman left a comment

Choose a reason for hiding this comment

jdhoffa commented Jul 18, 2024 • edited Loading

config: point `pa2024ch` to data prep with new weo 2023 scenarios #333

config: point `pa2024ch` to data prep with new weo 2023 scenarios #333

jdhoffa commented Jul 1, 2024 •

edited

Loading

github-actions bot commented Jul 2, 2024 •

edited

Loading

jdhoffa commented Jul 2, 2024 •

edited

Loading

jdhoffa commented Jul 2, 2024 •

edited

Loading

jdhoffa commented Jul 3, 2024 •

edited

Loading

jdhoffa commented Jul 18, 2024 •

edited

Loading