Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom gem repository not working with jruby-gradle-plugin 2.0.1 (plain command line gem does work with the same custom repo in .gemrc) #429

Open
tkrah opened this issue Oct 5, 2021 · 5 comments
Open

Custom gem repository not working with jruby-gradle-plugin 2.0.1 (plain command line gem does work with the same custom repo in .gemrc) #429

tkrah opened this issue Oct 5, 2021 · 5 comments

Comments

@tkrah
Copy link

tkrah commented Oct 5, 2021 •
edited
Loading

Hi,

I tried to get a custom gem repository working with jRuby but it does not work like described.

I did configure my local .gemrc with that URL which it should use (custom mirror - artifactory) and it works just fine with "gem install ..." etc. - so the upstream mirror is ok.

I choose to modify the example from the project:

https://github.com/jruby-gradle/jruby-gradle-plugin/tree/master/examples/run-script-with-jruby-args

and modified the repository line from

ruby.gems()

to

ruby.gems("https://tkrah:[email protected]/artifactory/api/gems/gems-repos/")

Running the gradle task runGradleTest now just results in this output instead of a successful build:

exception thrown for request to /rubygems/colorize/
com.github.jrubygradle.api.core.ApiException: Count not retrieve list of versions for colorize
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.allVersions(DefaultRubyGemRestApi.groovy:97)
        at com.github.jrubygradle.internal.core.AbstractIvyXmlProxyServer.getDirectoryListing(AbstractIvyXmlProxyServer.groovy:181)
        at com.github.jrubygradle.internal.core.IvyXmlRatpackProxyServer.lambda$run$2(IvyXmlRatpackProxyServer.java:98)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.MethodHandler.handle(MethodHandler.java:44)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.DefaultContext.insert(DefaultContext.java:179)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:61)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:63)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:63)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.DefaultContext.lambda$start$4(DefaultContext.java:104)
        at ratpack.exec.internal.DefaultExecution.lambda$new$0(DefaultExecution.java:82)
        at ratpack.exec.internal.DefaultExecution$InitialExecStream.exec(DefaultExecution.java:403)
        at ratpack.exec.internal.DefaultExecution.exec(DefaultExecution.java:224)
        at ratpack.exec.internal.DefaultExecution.exec(DefaultExecution.java:217)
        at ratpack.exec.internal.DefaultExecution.drain(DefaultExecution.java:187)
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
        at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:73)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
        at ratpack.exec.internal.DefaultExecController$ExecControllerBindingThreadFactory.lambda$newThread$0(DefaultExecController.java:137)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: groovyx.net.http.HttpException: Not Found
        at groovyx.net.http.NativeHandlers.failure(NativeHandlers.java:69)
        at groovyx.net.http.HttpBuilder$ResponseHandlerFunction.apply(HttpBuilder.java:2305)
        at groovyx.net.http.OkHttpBuilder.execute(OkHttpBuilder.java:290)
        at groovyx.net.http.OkHttpBuilder.doGet(OkHttpBuilder.java:184)
        at groovyx.net.http.HttpObjectConfigImpl.nullInterceptor(HttpObjectConfigImpl.java:47)
        at groovyx.net.http.HttpBuilder.get(HttpBuilder.java:346)
        at groovyx.net.http.HttpBuilder.get(HttpBuilder.java:1297)
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.getData(DefaultRubyGemRestApi.groovy:159)
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.allVersions(DefaultRubyGemRestApi.groovy:95)
        ... 27 more
exception thrown for request to /rubygems/colorize/
com.github.jrubygradle.api.core.ApiException: Count not retrieve list of versions for colorize
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.allVersions(DefaultRubyGemRestApi.groovy:97)
        at com.github.jrubygradle.internal.core.AbstractIvyXmlProxyServer.getDirectoryListing(AbstractIvyXmlProxyServer.groovy:181)
        at com.github.jrubygradle.internal.core.IvyXmlRatpackProxyServer.lambda$run$2(IvyXmlRatpackProxyServer.java:98)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.MethodHandler.handle(MethodHandler.java:44)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.DefaultContext.insert(DefaultContext.java:179)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:61)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:63)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:63)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.DefaultContext.lambda$start$4(DefaultContext.java:104)
        at ratpack.exec.internal.DefaultExecution.lambda$new$0(DefaultExecution.java:82)
        at ratpack.exec.internal.DefaultExecution$InitialExecStream.exec(DefaultExecution.java:403)
        at ratpack.exec.internal.DefaultExecution.exec(DefaultExecution.java:224)
        at ratpack.exec.internal.DefaultExecution.exec(DefaultExecution.java:217)
        at ratpack.exec.internal.DefaultExecution.drain(DefaultExecution.java:187)
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
        at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:73)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
        at ratpack.exec.internal.DefaultExecController$ExecControllerBindingThreadFactory.lambda$newThread$0(DefaultExecController.java:137)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: groovyx.net.http.HttpException: Not Found
        at groovyx.net.http.NativeHandlers.failure(NativeHandlers.java:69)
        at groovyx.net.http.HttpBuilder$ResponseHandlerFunction.apply(HttpBuilder.java:2305)
        at groovyx.net.http.OkHttpBuilder.execute(OkHttpBuilder.java:290)
        at groovyx.net.http.OkHttpBuilder.doGet(OkHttpBuilder.java:184)
        at groovyx.net.http.HttpObjectConfigImpl.nullInterceptor(HttpObjectConfigImpl.java:47)
        at groovyx.net.http.HttpBuilder.get(HttpBuilder.java:346)
        at groovyx.net.http.HttpBuilder.get(HttpBuilder.java:1297)
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.getData(DefaultRubyGemRestApi.groovy:159)
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.allVersions(DefaultRubyGemRestApi.groovy:95)
        ... 27 more
exception thrown for request to /rubygems/colorize/
com.github.jrubygradle.api.core.ApiException: Count not retrieve list of versions for colorize
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.allVersions(DefaultRubyGemRestApi.groovy:97)
        at com.github.jrubygradle.internal.core.AbstractIvyXmlProxyServer.getDirectoryListing(AbstractIvyXmlProxyServer.groovy:181)
        at com.github.jrubygradle.internal.core.IvyXmlRatpackProxyServer.lambda$run$2(IvyXmlRatpackProxyServer.java:98)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.MethodHandler.handle(MethodHandler.java:44)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.DefaultContext.insert(DefaultContext.java:179)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:61)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:63)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.path.internal.PathHandler.handle(PathHandler.java:63)
        at ratpack.handling.internal.DefaultContext.next(DefaultContext.java:157)
        at ratpack.handling.internal.DefaultContext.lambda$start$4(DefaultContext.java:104)
        at ratpack.exec.internal.DefaultExecution.lambda$new$0(DefaultExecution.java:82)
        at ratpack.exec.internal.DefaultExecution$InitialExecStream.exec(DefaultExecution.java:403)
        at ratpack.exec.internal.DefaultExecution.exec(DefaultExecution.java:224)
        at ratpack.exec.internal.DefaultExecution.exec(DefaultExecution.java:217)
        at ratpack.exec.internal.DefaultExecution.drain(DefaultExecution.java:187)
        at io.netty.util.concurrent.PromiseTask$RunnableAdapter.call(PromiseTask.java:38)
        at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:73)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909)
        at ratpack.exec.internal.DefaultExecController$ExecControllerBindingThreadFactory.lambda$newThread$0(DefaultExecController.java:137)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: groovyx.net.http.HttpException: Not Found
        at groovyx.net.http.NativeHandlers.failure(NativeHandlers.java:69)
        at groovyx.net.http.HttpBuilder$ResponseHandlerFunction.apply(HttpBuilder.java:2305)
        at groovyx.net.http.OkHttpBuilder.execute(OkHttpBuilder.java:290)
        at groovyx.net.http.OkHttpBuilder.doGet(OkHttpBuilder.java:184)
        at groovyx.net.http.HttpObjectConfigImpl.nullInterceptor(HttpObjectConfigImpl.java:47)
        at groovyx.net.http.HttpBuilder.get(HttpBuilder.java:346)
        at groovyx.net.http.HttpBuilder.get(HttpBuilder.java:1297)
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.getData(DefaultRubyGemRestApi.groovy:159)
        at com.github.jrubygradle.internal.core.DefaultRubyGemRestApi.allVersions(DefaultRubyGemRestApi.groovy:95)
        ... 27 more
> Task :jrubyPrepare FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jrubyPrepare'.
> Could not resolve all files for configuration ':gems'.
   > Could not resolve rubygems:colorize:0.7.7+.
     Required by:
         project :
      > Failed to list versions for rubygems:colorize.
         > Could not list versions using Ivy pattern 'http://localhost:38737/[organisation]/[module]/[revision]/ivy.xml'.
            > Could not get resource 'http://localhost:38737/rubygems/colorize/'.
               > Could not GET 'http://localhost:38737/rubygems/colorize/'. Received status code 500 from server: Internal Server Error

So how do you use a custom repository with jRuby gradle plugin here? Just using the same URL which does work for command line running gem is not going to work here.
@tkrah tkrah changed the title Custom gem repository not working with jruby-gradle-plugin 2.0.1 (plain command line gem support is fine) Custom gem repository not working with jruby-gradle-plugin 2.0.1 (plain command line gem does work with the same custom repo in .gemrc) Oct 5, 2021
@tkrah tkrah mentioned this issue Oct 5, 2021
Open
@tkrah
Copy link
Author

tkrah commented Oct 7, 2021 •
edited
Loading

Some debugging revealed some interesting things - the actual request made is e.g.:

https://tkrah:[email protected]/api/v1/versions/colorize.json

This won't work - the repository does not run under that URL - it's missing the whole part about the gem-repos url part:

/artifactory/api/gems/gems-repos/

Additionally the OkHttpBuilder has a auth part called

applyAuth(requestBuilder, chainedConfig);

which has this:

    private static void applyAuth(final Request.Builder requestBuilder, final ChainedHttpConfig chainedConfig) {
        final HttpConfig.Auth auth = chainedConfig.getChainedRequest().actualAuth();
        if (auth != null) {
            switch (auth.getAuthType()) {
                case BASIC:
                    requestBuilder.addHeader("Authorization", Credentials.basic(auth.getUser(), auth.getPassword()));
                    break;
                case DIGEST:
                    // supported in constructor with an interceptor
            }
        }
    }

but authentication was not configured for that request (although it's included in the configured URL) - not sure if it will be respected or not, can't tell for sure because the whole URL does not match the configured one.

Maybe someone has an idea what's wrong.

@silverhammermba
Copy link

I have this problem as well. I'm using this plugin to run a Ruby script (with gems) as part of my gradle build. Everything works fine if I use the default ruby.gems() repository, but I can't find a way to tell it to use my artifactory instance instead.

Using bundler, I changed my Gemfile from source "https://rubygems.org" to source "https://myartifactory.jfrog.io/artifactory/api/gems/gems/" and stored my credentials in an environment variable

export BUNDLE_MYARTIFACTORY__JFROG__IO="user:pwd"

and that works great for bundler. I tried doing the obvious thing similar to @tkrah and changing the repo to ruby.gems("https://user:[email protected]/artifactory/api/gems/gems") but that fails to resolve with a similar error

404 client error for request to /rubygems/twine/1.1.1/ivy.xml

> Task :myproj:jrubyPrepare FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':myproj:jrubyPrepare'.
> Could not resolve all dependencies for configuration ':myproj:gems'.
   > Could not find rubygems:twine:1.1.1.
     Searched in the following locations:
       - http://localhost:61344/rubygems/twine/1.1.1/ivy.xml
       - https://myartifactory.jfrog.io/artifactory/libs-release/rubygems/twine/1.1.1/twine-1.1.1.pom
     Required by:
         project :myproj

@silverhammermba
Copy link

silverhammermba commented Nov 11, 2022 •
edited
Loading

I think the fact that it searched http://localhost:61344/rubygems/twine/1.1.1/ivy.xml is promising, because that's the internal ivy proxy that the jruby plugin sets up. The annoying part is I can't find any way to see how that proxy is trying to resolve my custom URI. So it's hard to debug.

@tkrah
Copy link
Author

tkrah commented Jan 11, 2023

@silverhammermba look at #429 (comment) - I've debugged through the code - the whole url resolution is wrong if you specify a custom repository and auth is unconfigured.

@raelik
Copy link
Contributor

raelik commented Aug 6, 2024

As far as I can tell, this fails because the authentication is never configured here:

jruby-gradle-plugin/core-plugin/src/main/groovy/com/github/jrubygradle/internal/core/DefaultRubyGemRestApi.groovy

Lines 167 to 175 in 024713c

private static HttpBuilder getHttpBuilder(URI uri) {
configure {
request.uri = uri
client.clientCustomizer { OkHttpClient.Builder builder ->
builder.followRedirects(true)
builder.followSslRedirects(true)
}
}
}

I believe fixing this would require either parsing the credentials out of the URI, or providing them from an external file or environment variable (I would probably allow for all 3 options) in the two DefaultRubyGemRestApi constructors, and creating a new function signature like this:

    private static HttpBuilder getHttpBuilder(URI uri, String username, String password) {
        configure {
            request.uri = uri
            request.auth.basic username, password
            client.clientCustomizer { OkHttpClient.Builder builder ->
                builder.followRedirects(true)
                builder.followSslRedirects(true)
            }
        }
    }

The constructors would call this signature is authentication was provided. There's probably a cleaner way to fix this, but that's the first way I could think of.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@raelik @silverhammermba @tkrah