S3 Compatible Storage from Dell Unable to Connect

[atharmahboob]

I am using Percona Everest 0.10.1. Installation using an RKE-1 kubernetes cluster set up over a bunch of OpenStack VMs established using Rancher went successfully.

Percona Everest services are running successfully as I am able to get into Percona Everest web interface successfully.

When I try to create S3 compatible storage it gives error "Unable to connect. Check your credentials".

I am using settings with: (1) Using Path Style URL access; and (2) Disable Verify TLS Certificate

However, with the same set of credentials and same connection settings I am able to connect and successfully work with the same storage using S3 client tools like s3cmd and s3-explorer.

The S3 storage backend is Dell Storage.

[recharte]

Hi @atharmahboob thank you for reporting this.
Can you share the logs of the Percona Everest API server pod?
kubectl logs pods/percona-everest-xxxxxxxxxx-xxxxx -n everest-system

[atharmahboob]

This is the log entry I get when trying to connect:

{"level":"error","T":"2024-06-04T21:36:09Z","caller":"api/validation.go:209","msg":"RequestError: send request failed\ncaused by: Head "https://s3.rapidcompute.com:80/test-bucket\": http: server gave HTTP response to HTTPS client","stacktrace":"github.com/percona/everest/api.s3Access\n\t/home/runner/work/everest/everest/api/validation.go:209\ngithub.com/percona/everest/api.validateStorageAccessByCreate\n\t/home/runner/work/everest/everest/api/validation.go:161\ngithub.com/percona/everest/api.validateCreateBackupStorageRequest\n\t/home/runner/work/everest/everest/api/validation.go:420\ngithub.com/percona/everest/api.(*EverestServer).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/backup_storage.go:71\ngithub.com/percona/everest/api.(*ServerInterfaceWrapper).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/everest-server.gen.go:1478\ngithub.com/oapi-codegen/echo-middleware.OapiRequestValidatorWithOptions.func1.1\n\t/home/runner/go/pkg/mod/github.com/oapi-codegen/[email protected]/oapi_validate.go:105\ngithub.com/percona/everest/api.(*EverestServer).authenticate-fm.(*EverestServer).authenticate.func1\n\t/home/runner/work/everest/everest/api/auth.go:51\ngithub.com/labstack/echo/v4.(*Echo).add.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:582\ngithub.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/logger.go:126\ngithub.com/percona/everest/api.NewEverestServer.RateLimiter.RateLimiterWithConfig.func1.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/rate_limiter.go:148\ngithub.com/labstack/echo/v4.(*Echo).ServeHTTP.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:663\ngithub.com/percona/everest/api.(*EverestServer).initHTTPServer.RemoveTrailingSlash.RemoveTrailingSlashWithConfig.func5.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/slash.go:118\ngithub.com/labstack/echo/v4.(*Echo).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:3137\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:2039"}
ubuntu@k8s-dn:~$

[atharmahboob]

And this :

{"level":"error","T":"2024-06-04T21:42:25Z","caller":"api/validation.go:209","msg":"RequestError: send request failed\ncaused by: Head "https://s3.rapidcompute.com:80/test-bucket\": http: server gave HTTP response to HTTPS client","stacktrace":"github.com/percona/everest/api.s3Access\n\t/home/runner/work/everest/everest/api/validation.go:209\ngithub.com/percona/everest/api.validateStorageAccessByCreate\n\t/home/runner/work/everest/everest/api/validation.go:161\ngithub.com/percona/everest/api.validateCreateBackupStorageRequest\n\t/home/runner/work/everest/everest/api/validation.go:420\ngithub.com/percona/everest/api.(*EverestServer).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/backup_storage.go:71\ngithub.com/percona/everest/api.(*ServerInterfaceWrapper).CreateBackupStorage\n\t/home/runner/work/everest/everest/api/everest-server.gen.go:1478\ngithub.com/oapi-codegen/echo-middleware.OapiRequestValidatorWithOptions.func1.1\n\t/home/runner/go/pkg/mod/github.com/oapi-codegen/[email protected]/oapi_validate.go:105\ngithub.com/percona/everest/api.(*EverestServer).authenticate-fm.(*EverestServer).authenticate.func1\n\t/home/runner/work/everest/everest/api/auth.go:51\ngithub.com/labstack/echo/v4.(*Echo).add.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:582\ngithub.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/logger.go:126\ngithub.com/percona/everest/api.NewEverestServer.RateLimiter.RateLimiterWithConfig.func1.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/rate_limiter.go:148\ngithub.com/labstack/echo/v4.(*Echo).ServeHTTP.func1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:663\ngithub.com/percona/everest/api.(*EverestServer).initHTTPServer.RemoveTrailingSlash.RemoveTrailingSlashWithConfig.func5.1\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/slash.go:118\ngithub.com/labstack/echo/v4.(*Echo).ServeHTTP\n\t/home/runner/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669\nnet/http.serverHandler.ServeHTTP\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:3137\nnet/http.(*conn).serve\n\t/opt/hostedtoolcache/go/1.22.3/x64/src/net/http/server.go:2039"}

[atharmahboob]

Here is a screenshot of my connection settings:

[atharmahboob]

I think Percon a Everest is constructing the connection URL with https:// instead of http// despite my selecting a non-TLS connection.

Please advise.

[recharte]

@atharmahboob if you want to connect to your storage server using http instead of https you should explicitly set it in the endpoint URL like so http://s3.rapidcompute.com.

Disabling the verify TLS certificate option doesn't assume http. Instead, it connects using https but skips the certificate validation. The intended use case for unsetting this option is when the storage server is using a self-signed certificate but you still you want to connect using https.