Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove antlr from dependency from java-core pom #33030

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Abacn
Copy link
Contributor

@Abacn Abacn commented Nov 6, 2024

Alternative fix of #32696

We already vendored and repackaged antlr runtime into the jar. It was intended to avoid dependency version conflict if user using different antlr versions. However, beam java core still declares it as a dependency.

Before:

java core pom has antlr4-runtime dependency, e.g. in https://repo1.maven.org/maven2/org/apache/beam/beam-sdks-java-core/2.60.0/beam-sdks-java-core-2.60.0.pom

<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
<version>4.7</version>
<scope>compile</scope>

after:

ran ./gradlew :sdks:java:core:publishToMavenLocal -Ppublishing and checked that antlr-runtime is no longer a dependency of beam java core in generated pom.

Also checked the jar still repackaged antlr runtime in org/apache/beam/repackaged/core/org/antlr/v4/runtime

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@github-actions github-actions bot added the java label Nov 6, 2024
@pinfrederic
Copy link

I think that solution is even better than the previous one.
I guess this time we can use a different version in our pipelines if we need it... can you confirm?
With this solution do we still have a risk of having conflicts with beam at runtime?

@Abacn
Copy link
Contributor Author

Abacn commented Nov 6, 2024

Thanks for the feedback.

I guess this time we can use a different version in our pipelines if we need it... can you confirm?

Yes. I think with current version user can still use a different version, but they need to pin the dependency like #32696 (comment)

The problem on the current HEAD is that beam core "leaks" the antlr-runtime dependency while it was supposed to be shaded

do we still have a risk of having conflicts with beam at runtime?

Yes that is what I understood.

@Abacn
Copy link
Contributor Author

Abacn commented Nov 6, 2024

R: @shunping

@Abacn Abacn changed the title Remove antlr from dependency of jaca-core pom Remove antlr from dependency of java-core pom Nov 6, 2024
@Abacn Abacn changed the title Remove antlr from dependency of java-core pom Remove antlr from dependency from java-core pom Nov 6, 2024
Copy link
Contributor

github-actions bot commented Nov 6, 2024

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@pinfrederic
Copy link

#Yes. I think with current version user can still use a different version, but they need to pin the dependency like #32696 (comment)

Can you set the dependency as "compile only" in the gradle file? This would avoid us to do this
exclude group: 'org.antlr', module: '*'

@Abacn
Copy link
Contributor Author

Abacn commented Nov 6, 2024

#Yes. I think with current version user can still use a different version, but they need to pin the dependency like #32696 (comment)

Can you set the dependency as "compile only" in the gradle file? This would avoid us to do this exclude group: 'org.antlr', module: '*'

After this change antlr will be a "compile only" dependency (more precisely its a custom gradle config, used for generating java code from antlr source), won't leak into the pom.

do we still have a risk of having conflicts with beam at runtime?

Yes that is what I understood.

I mean there could be other Beam's dependency, not Beam itself, having transient antlr dependency, that could be conflict with user dependency. This is based on the observation of #33016 #14787 - bumping antlr version then some tests breaks elsewhere.

@pinfrederic
Copy link

Alright, understood. Thanks

@Abacn
Copy link
Contributor Author

Abacn commented Nov 6, 2024

many tests failing in ':runners:direct-java:needsRunnerTests' target due to

java.lang.NoClassDefFoundError: org/antlr/v4/runtime/CharStream
	at org.apache.beam.sdk.schemas.FieldAccessDescriptor.withFieldNames(FieldAccessDescriptor.java:187)
	at org.apache.beam.sdk.schemas.FieldAccessDescriptor.withFieldNames(FieldAccessDescriptor.java:174)

checked that the same test, if running under ':runners:direct-java:test', will pass.

The reason it failed is because the tests running on unshaded compiled class (haven't redirect antlr yet)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants