Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI for forward-porting GC3 patches to GC4 #147

Draft
wants to merge 11 commits into
base: gc4
Choose a base branch
from

Conversation

ddeclerck
Copy link
Contributor

Follow-up of #146.

@ddeclerck ddeclerck changed the title Dummy commit CI for forward-porting GC3 patches to GC4 May 29, 2024
@ddeclerck ddeclerck force-pushed the gc3_to_gc4 branch 3 times, most recently from 6900d8d to 1c95bd0 Compare May 30, 2024 21:21
@@ -96,12 +96,15 @@ autoreconf $AC_OPTS $MAINPATH > $msgs 2>&1; ret=$?
# Filter aminclude_static as those are only used _within_ another
# check so reporting as portability problem is only noise.
# This has the effect of redirecting some error messages to stdout.
# to be moved to the Makefile - currently only usable for bootstrap,
# but should be done on autogen, too
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A, that old TODO...

Copy link
Collaborator

@GitMensch GitMensch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to wrap the commits again. From what I've inspected we need one refactor for integrating 4.x logic (you've spotted that well) nicely.

libcob/fileio.c Outdated
snprintf (file_open_env, (size_t)COB_FILE_MAX, "%s%s", "IO_", s);
if ((file_open_io_env = cob_get_env (file_open_env, NULL)) == NULL) {
snprintf (file_open_env, (size_t)COB_FILE_MAX, "%s%s", "io_", s);
if (f != NULL) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When/why should f be null here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because some functions (open_cbl_file, cob_sys_delete_file, ...) were "improved" to perform file mapping in GC3 (rev 3944), by calling the cob_chk_file_mapping function, which does not take a cob_file argument in GC3 but does in GC4, and that function in turn calls cob_chk_file_env. Since these functions (open_cbl_file, cob_sys_delete_file, ...) do not use a cob_file object, I resorted to passing NULL and coping with that...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this okay for you @GitMensch ?

libcob/fileio.c Outdated Show resolved Hide resolved
@GitMensch
Copy link
Collaborator

oops, hope I haven't broken the gitignore in f36dcda - if not then we likely should apply that to the gcos3x branch as well.

@ddeclerck
Copy link
Contributor Author

ddeclerck commented May 31, 2024

I suggest to wrap the commits again. From what I've inspected we need one refactor for integrating 4.x logic (you've spotted that well) nicely.

Saw your message a bit late, added another commit in the meantime 😅

By wrapping up you mean, committing to SVN ? (after doing the requested modifications of course)

@GitMensch
Copy link
Collaborator

GitMensch commented Jun 1, 2024 via email

@ddeclerck
Copy link
Contributor Author

I tend to be overly "conservative". Indeed this piece of code is barely modified afterwards, so I'll do the refactoring.

@ddeclerck
Copy link
Contributor Author

Is this okay to merge (@GitMensch) ?

@GitMensch
Copy link
Collaborator

I'll try to review this (late) evening.

@GitMensch
Copy link
Collaborator

looks_absolute should use "src", not file_open_name directly (merge issue?)

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

@ddeclerck
Copy link
Contributor Author

looks_absolute should use "src", not file_open_name directly (merge issue?)

This change is introduced in a later commit (3993).

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

By "new use", de you mean the fact that we apply file paths to the complex case ?

@GitMensch
Copy link
Collaborator

By "new use", de you mean the fact that we apply file paths to the complex case ?

yes

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

Depends on how other functions do it - it is best for now to mimic that (once the merge is completed we may revisit that part, but there are "some" commits left until we get there).

@ddeclerck
Copy link
Contributor Author

I made the necessary changes.

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

I find it more convenient to merge consecutive commits. If that's okay for you I could add to the current batch the next eligible commits until 3993 (that would be 6 commits: 3973, 3979, 3988, 3989, 3992 and 3993).

@GitMensch
Copy link
Collaborator

That batch is good to go :-)

@ddeclerck
Copy link
Contributor Author

That batch is good to go :-)

Merged in SVN ;)

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

@GitMensch
Copy link
Collaborator

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

No, only new files are copied, the others left as-is; before a release I regenerate the files but the files are nearly completely maintained by the translation project.
So just record the merge, then copy over new files and svn add them, if there are any.

And of course

@ddeclerck
Copy link
Contributor Author

ddeclerck commented Jun 8, 2024

Alright.

And of course

Hope this unfinished sentence did not have any vital info 😅

@GitMensch
Copy link
Collaborator

I can't remember any important info missing there.

@codecov-commenter
Copy link

codecov-commenter commented Jun 8, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 69.42446% with 170 lines in your changes missing coverage. Please review.

Please upload report for BASE (gc4@caeacd5). Learn more about missing BASE report.

Files with missing lines Patch % Lines
cobc/typeck.c 69.51% 49 Missing and 26 partials ⚠️
cobc/tree.c 64.48% 11 Missing and 27 partials ⚠️
cobc/codegen.c 64.28% 27 Missing and 8 partials ⚠️
libcob/common.c 84.93% 7 Missing and 4 partials ⚠️
cobc/parser.y 54.54% 6 Missing and 4 partials ⚠️
cobc/config.c 66.66% 0 Missing and 1 partial ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@          Coverage Diff           @@
##             gc4     #147   +/-   ##
======================================
  Coverage       ?   65.72%           
======================================
  Files          ?       37           
  Lines          ?    65593           
  Branches       ?    18281           
======================================
  Hits           ?    43113           
  Misses         ?    15457           
  Partials       ?     7023           
Flag Coverage Δ
65.72% <69.42%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ddeclerck
Copy link
Contributor Author

Quick question: I sometimes see alternative code for GC4 in #if 0 blocks; I guess I should implement those and drop the other branch, correct ?

I'm talking about those:

#if 0 /* TODO for 4.0: set the attributes from the field given outside on the stack */
		output ("cob_field *cob_fret, const int cob_pam");
#else
		output ("cob_field **cob_fret, const int cob_pam");
#endif

@GitMensch
Copy link
Collaborator

That's quite a bunch - any reason to not merge upstream?
Open issues you are aware of or special adjustments needed?

[we really need to get to commits that have someone else in the ChangeLogs...]

@ddeclerck
Copy link
Contributor Author

ddeclerck commented Jun 12, 2024

That's quite a bunch - any reason to not merge upstream? Open issues you are aware of or special adjustments needed?

No good reason. It may be many commits, but the first batch had way more lines (this one is only +1,162 −904).
Anyways, I'll merge upstream if that's okay with you.

[we really need to get to commits that have someone else in the ChangeLogs...]

I'm looking forward to reaching commit 4614 - our first contribution to GC3 ;)

@GitMensch
Copy link
Collaborator

GitMensch commented Oct 7, 2024

I'm confused - that is an index so and as long as it isn't used as subscript or SET there should be no check for that.
Checking... this is cob_check_subscript() generated (only checked GC3), which checks and sets EC-BOUNDS-SUBSCRIPT.

Per standard that would be raised for its actual use in a subscript with an invalid value or if an arithmetic expression in SET does not result in an integer.

If it would be out of the limits of its matching OCCURS clause (which is the case here) then it should raise EC-RANGE-INDEX instead (still fatal)... but we don't have a SET statement for this.

Checking further: COBOL2002 (which added exceptions) added to the OCCURS clause the EC-RANGE-INDEX and that is to be set if the value is higher than the implementor defined maximum range, which should at least include the range "1 - occurs-to-integer" to "occurs-to-integer * 2" (no idea how that specific value got in and interesting to see that it allows for negative values as well). This is to be applied by adjustments during PERFORM VARYING (yay, our case).

As our implementor range is an int that should not happen in this case.

So: it does seem that the whole generated check is wrong - that should only be done when the index is used within a subscript.
A generation for EC-RANGE-INDEX could be minimal useful (as noted, it would compare against INT_MIN and INT_MAX) and apply in the PERFORM statement.

Please check if that's already bad in GC 3.3 and adjust your fix for GC4.

@ddeclerck
Copy link
Contributor Author

So: it does seem that the whole generated check is wrong - that should only be done when the index is used within a subscript.

So, does that mean this bound check added (only) to 4.x by Ron in 4953 should in fact be removed ?

Other than this, there is already a check both in 3.x and 4.x when the index is actually used.

@GitMensch
Copy link
Collaborator

So: it does seem that the whole generated check is wrong - that should only be done when the index is used within a subscript.

So, does that mean this bound check added (only) to 4.x by Ron in 4953 should in fact be removed ?

Other than this, there is already a check both in 3.x and 4.x when the index is actually used.

I see now where the issue is...

The idea of that commit:

  • in general: add some reasonable warnings previously missing
  • in case of an index: move the checking to a place that's much less often done than accessing the index: the time where it is set

So the things that need to be done in a separate commit, ideally up-front upstream:

  • use your adjusted codegen
  • keep all the warnings of the mentioned commit and, if not already done, also use them for subscripts (in the field resolving by cb_ref -> apply if the subscript (or the start / len of ref-mod) are only a single literal
  • have the old codegen by default (that includes no checking during PERFORM/SET)
  • add a new flag -fopt-check-subscript-set that skips the subscript check for the hasval case (renaming that to known_value or similar would be useful) and do it (non-standard) during PERFORM and SET (which seems to be #ifdef 0'd) [still only generated if ec-bounds-subscript is active]

As that new flag will only be used in its specific test case (maybe compile and run w/wo) it won't break NIST any more, but provides a possible less-expensive way to check use of index-variables in production.

What do you think?

Note: "the future"TM should bring an adjusted codegen that has the subscript check for INDEXED BY (or constant expressions containing it) only once:

101   MOVE TAB-F1(IDX) TO VAR
102   MOVE TAB-F1(IDX) TO TAB-F1(IDX+1), TAB-F2(IDX+1)
103   PERFORM SOMETHING.
104   IF TAB-F2 (IDX) NOT = VAR2
105      MOVE TAB-F2(IDX) TO VAR2, VAR3.
106   SET IDX UP BY 1
107   MOVE TAB-F1(IDX) TO TAB-F3(IDX) 

here the runtime check, which currently happens in all those cases) would be

  • done on line 101
  • skipped on line 102 for the left side
  • done on line 101 for the first target - because that's an expression, skipped for the second as the expression is identical
  • done on line 104 because the PERFORM may have changed that (same would be true if a label was defined)
  • skipped on line 105
  • done on line 107 as changed in line 106
  • skipped on line 107 for the right side

Even then the new -fopt-check-subscript-set can bring in some performance (but much less than it can now).

@ddeclerck
Copy link
Contributor Author

So the things that need to be done in a separate commit, ideally up-front upstream:

  • use your adjusted codegen
  • keep all the warnings of the mentioned commit and, if not already done, also use them for subscripts (in the field resolving by cb_ref -> apply if the subscript (or the start / len of ref-mod) are only a single literal
  • have the old codegen by default (that includes no checking during PERFORM/SET)
  • add a new flag -fopt-check-subscript-set that skips the subscript check for the hasval case (renaming that to known_value or similar would be useful) and do it (non-standard) during PERFORM and SET (which seems to be #ifdef 0'd) [still only generated if ec-bounds-subscript is active]

As that new flag will only be used in its specific test case (maybe compile and run w/wo) it won't break NIST any more, but provides a possible less-expensive way to check use of index-variables in production.

What do you think?

That seems reasonable. Should this be done in 3.x too, or just in 4.x ?

@GitMensch
Copy link
Collaborator

I'm always happy if someone merges from 4 to 3, then do fixes there and merge it back to 4 :-)

@ddeclerck
Copy link
Contributor Author

I'm always happy if someone merges from 4 to 3, then do fixes there and merge it back to 4 :-)

I guess I'll go for it then ;)

@ddeclerck
Copy link
Contributor Author

Actually just realizing... Doesn't that interfere with 3.x's subscript-check dialect option ?

@GitMensch
Copy link
Collaborator

Possibly - can you please elaborate? Note: neither the warning part nor the extra -fopt would per default conflict with a dialect setting.

@ddeclerck
Copy link
Contributor Author

Possibly - can you please elaborate? Note: neither the warning part nor the extra -fopt would per default conflict with a dialect setting.

I made a PR to discuss this: #191.

@GitMensch
Copy link
Collaborator

GitMensch commented Oct 10, 2024

The alternative is to merge the changes and move the new non-iso code into a #if 0 /* FIXME: add back as option, because not conforming to ISO- disabling it and make a TODO entry about "needs to be re-enabled but then as separate option and also included into the new dialect config". I don't think Ron would be lucky about that... but I'd say for the benefit of "rest is merged" disabling this part "for now" is acceptable.

But if you do that then please make that (the disabling + TODO entry + moving the syntax check to syn_occurs.at and make it an XFAIL if you can't keep the warning) a separate commit before the merge ones titled something like "disable r4953 index-check optimization to allow merging (should be enabled later as an optional flag)" or similar.

@ddeclerck
Copy link
Contributor Author

But if you do that then please make that (the disabling + TODO entry + moving the syntax check to syn_occurs.at and make it an XFAIL if you can't keep the warning) a separate commit before the merge ones titled something like "disable r4953 index-check optimization to allow merging (should be enabled later as an optional flag)" or similar.

You mean a direct commit into SVN trunk ?

As for "moving the syntax check to syn_occurs.at", the new test introduced by 4953 in run_subscripts.at both has a compile-time check and a runtime check. Or were you referring to some other test ?

@GitMensch
Copy link
Collaborator

You mean a direct commit into SVN trunk ?

yes, your merge here would then be based on that

... moving the syntax check to syn_occurs.at

That would mean (either with the disabling option or with the integration option to GC3) to change
run_subscripts.at addition from r4953 to:

  • copy the test to syn_occurs.at, dropping the SUBN part as compile-time verification
  • ideally adjust the parser to only allow numeric-sources for the SET index-name and SET pointer-name (the program should not compile), just testing for the type and raise "must be numeric" and for non-integer like 0.2 "must be an integer" - that should not get into typeck.c where the current warning is
  • reduce that big COMPILE line to the minimal version $COMPILE prog.cob and drop the parts now raised as error parts.

@ddeclerck
Copy link
Contributor Author

ddeclerck commented Nov 7, 2024

Hi @GitMensch ,
As you requested smaller batches, I think this one should be good enough (not many commits, but many lines already).
It notably includes the PICTURE P fix from GC3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants