Primary metastore does not have database #189

chengang2 · 2020-02-13T08:26:07Z

when I run hive --hiveconf hive.metastore.uris=****
waggle-dance error :


org.apache.hadoop.hive.metastore.api.NoSuchObjectException: Primary metastore does not have database @hive#ad_test
        at com.hotels.bdp.waggledance.mapping.service.impl.StaticDatabaseMappingService.databaseMapping(StaticDatabaseMappingService.java:268) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.mapping.service.impl.MonitoredDatabaseMappingService.databaseMapping(MonitoredDatabaseMappingService.java:48) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_tables_by_type_aroundBody614(FederatedHMSHandler.java:1644) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure615.run(FederatedHMSHandler.java:1) ~[waggle-dance-core-3.5.0.jar!/:?]
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.8.13.jar!/:1.8.13]
        at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:58) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:47) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_tables_by_type_aroundBody616(FederatedHMSHandler.java:1642) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure617.run(FederatedHMSHandler.java:1) ~[waggle-dance-core-3.5.0.jar!/:?]
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.8.13.jar!/:1.8.13]
        at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:213) ~[jcabi-aspects-0.22.6.jar!/:?]
        at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) ~[jcabi-aspects-0.22.6.jar!/:?]
        at com.jcabi.aspects.aj.MethodLogger.wrapMethod(MethodLogger.java:169) ~[jcabi-aspects-0.22.6.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_tables_by_type(FederatedHMSHandler.java:1642) ~[waggle-dance-core-3.5.0.jar!/:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221]
        at com.hotels.bdp.waggledance.server.ExceptionWrappingHMSHandler.invoke(ExceptionWrappingHMSHandler.java:50) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.sun.proxy.$Proxy137.get_tables_by_type(Unknown Source) ~[?:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221]
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at com.sun.proxy.$Proxy137.get_tables_by_type(Unknown Source) ~[?:?]
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_tables_by_type.getResult(ThriftHiveMetastore.java:11339) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_tables_by_type.getResult(ThriftHiveMetastore.java:11323) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) [libthrift-0.9.3.jar!/:0.9.3]
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) [libthrift-0.9.3.jar!/:0.9.3]
        at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48) [hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) [libthrift-0.9.3.jar!/:0.9.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_221]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_221]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_221]

this my conf:

primary-meta-store:
  access-control-type: READ_AND_WRITE_AND_CREATE_ON_DATABASE_WHITELIST
  database-prefix: ad_
  latency: 0
  mapped-databases:
  - ad_test
  - ad_test2
  name: primary
  remote-meta-store-uris: thrift://cg-test-mysql-0.cg-test-mysql-headless.default.svc.cluster.local:9083
  writable-database-white-list:
  - ad_test
  - ad_test2
federated-meta-stores:
- access-control-type: READ_AND_WRITE_ON_DATABASE_WHITELIST
  database-prefix: cg
  latency: 0
  mapped-databases:
  - cgtest
  name: federated
  remote-meta-store-uris: thrift://cg-test-mysql-1.cg-test-mysql-headless.default.svc.cluster.local:9083
  writable-database-white-list:
  - cgtest

The text was updated successfully, but these errors were encountered:

chengang2 · 2020-02-13T08:27:54Z

service start ok,bug can not run . hive error :
hive> use ad_test
> ;
FAILED: SemanticException [Error 10072]: Database does not exist: ad_test

massdosage · 2020-02-13T11:58:36Z

Why do you have ad_test and ad_test2 configured twice? Can you also please reformat the config using backticks so it renders exactly as it's written, it's hard to tell what's going on above as Markdown is rendering things as bulleted lists.

patduin · 2020-02-13T12:17:26Z

Try to set the waggle-dance-server.yml: database-resolution: PREFIXED
As it look from your config that's what you want to do. Then try again.
Also possible remove:

mapped-databases:
  - ad_test
  - ad_test2

To simplify the config.
The error in WD and in your Hive indicates that ad_test database just doesn't exist. What does show databases; tell you?

chengang2 · 2020-02-13T13:17:57Z

@patduin cat waggle-dance-federation.yml
primary-meta-store:
access-control-type: READ_ONLY
database-prefix: ''
latency: 0
name: primary
remote-meta-store-uris: thrift://cg-test-mysql-0.cg-test-mysql-headless.default.svc.cluster.local:9083
I encountered the same error：
ERROR 6266 --- [pool-4-thread-1] o.a.t.ProcessFunction : Internal error processing get_tables_by_type

org.apache.hadoop.hive.metastore.api.NoSuchObjectException: Primary metastore does not have database @hive#ad_test

hive show databases：
hive> show databases;
OK
ad_test
ad_test2
cg_test
cg_test2
cgtest3
default
Time taken: 0.76 seconds, Fetched: 6 row(s)
hive> use ad_test;
FAILED: SemanticException [Error 10072]: Database does not exist: ad_test

chengang2 · 2020-02-13T13:20:14Z

@massdosage cat waggle-dance-federation.yml
primary-meta-store:
access-control-type: READ_ONLY
database-prefix: ''
latency: 0
name: primary
remote-meta-store-uris: thrift://cg-test-mysql-0.cg-test-mysql-headless.default.svc.cluster.local:9083

I encountered the same error：
ERROR 6266 --- [pool-4-thread-1] o.a.t.ProcessFunction : Internal error processing get_tables_by_type

org.apache.hadoop.hive.metastore.api.NoSuchObjectException: Primary metastore does not have database @hive#ad_test

hive show databases：
hive> show databases;
OK
ad_test
ad_test2
cg_test
cg_test2
cgtest3
default
Time taken: 0.76 seconds, Fetched: 6 row(s)
hive> use ad_test;
FAILED: SemanticException [Error 10072]: Database does not exist: ad_test

patduin · 2020-02-13T15:30:04Z

Try to set the waggle-dance-server.yml: database-resolution: PREFIXED

Also can you attach the full logs?

chengang2 · 2020-02-14T03:53:09Z

@patduin set the waggle-dance-server.yml: database-resolution: PREFIXED

2020-02-14 03:49:46.593 ERROR 7206 --- [pool-4-thread-1] o.a.h.h.m.RetryingHMSHandler             : java.lang.IllegalArgumentException
        at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1314)
        at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1202)
        at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:89)
        at com.hotels.bdp.waggledance.mapping.service.PanopticConcurrentOperationExecutor.executeRequests(PanopticConcurrentOperationExecutor.java:47)
        at com.hotels.bdp.waggledance.mapping.service.PanopticOperationHandler.getAllDatabases(PanopticOperationHandler.java:77)
        at com.hotels.bdp.waggledance.mapping.service.impl.PrefixBasedDatabaseMappingService$1.getAllDatabases(PrefixBasedDatabaseMappingService.java:278)
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_databases_aroundBody26(FederatedHMSHandler.java:236)
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure27.run(FederatedHMSHandler.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
        at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:58)
        at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:47)
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_databases_aroundBody28(FederatedHMSHandler.java:235)
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure29.run(FederatedHMSHandler.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
        at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:213)
        at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1)
        at com.jcabi.aspects.aj.MethodLogger.wrapMethod(MethodLogger.java:169)
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_databases(FederatedHMSHandler.java:235)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.hotels.bdp.waggledance.server.ExceptionWrappingHMSHandler.invoke(ExceptionWrappingHMSHandler.java:50)
        at com.sun.proxy.$Proxy137.get_databases(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
        at com.sun.proxy.$Proxy137.get_databases(Unknown Source)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_databases.getResult(ThriftHiveMetastore.java:10803)
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_databases.getResult(ThriftHiveMetastore.java:10787)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

2020-02-14 03:49:46.593 ERROR 7206 --- [pool-4-thread-1] o.a.t.s.TThreadPoolServer                : Error occurred during processing of message.

java.lang.IllegalArgumentException: null
        at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1314) ~[?:1.8.0_221]
        at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1202) ~[?:1.8.0_221]
        at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:89) ~[?:1.8.0_221]
        at com.hotels.bdp.waggledance.mapping.service.PanopticConcurrentOperationExecutor.executeRequests(PanopticConcurrentOperationExecutor.java:47) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.mapping.service.PanopticOperationHandler.getAllDatabases(PanopticOperationHandler.java:77) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.mapping.service.impl.PrefixBasedDatabaseMappingService$1.getAllDatabases(PrefixBasedDatabaseMappingService.java:278) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_databases_aroundBody26(FederatedHMSHandler.java:236) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure27.run(FederatedHMSHandler.java:1) ~[waggle-dance-core-3.5.0.jar!/:?]
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.8.13.jar!/:1.8.13]
        at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:58) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:47) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_databases_aroundBody28(FederatedHMSHandler.java:235) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure29.run(FederatedHMSHandler.java:1) ~[waggle-dance-core-3.5.0.jar!/:?]
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.8.13.jar!/:1.8.13]
        at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:213) ~[jcabi-aspects-0.22.6.jar!/:?]
        at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) ~[jcabi-aspects-0.22.6.jar!/:?]
        at com.jcabi.aspects.aj.MethodLogger.wrapMethod(MethodLogger.java:169) ~[jcabi-aspects-0.22.6.jar!/:?]
        at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_databases(FederatedHMSHandler.java:235) ~[waggle-dance-core-3.5.0.jar!/:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221]
        at com.hotels.bdp.waggledance.server.ExceptionWrappingHMSHandler.invoke(ExceptionWrappingHMSHandler.java:50) ~[waggle-dance-core-3.5.0.jar!/:?]
        at com.sun.proxy.$Proxy137.get_databases(Unknown Source) ~[?:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_221]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_221]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_221]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_221]
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at com.sun.proxy.$Proxy137.get_databases(Unknown Source) ~[?:?]
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_databases.getResult(ThriftHiveMetastore.java:10803) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_databases.getResult(ThriftHiveMetastore.java:10787) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[libthrift-0.9.3.jar!/:0.9.3]
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[libthrift-0.9.3.jar!/:0.9.3]
        at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48) ~[hive-metastore-2.3.3.jar!/:2.3.3]
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) [libthrift-0.9.3.jar!/:0.9.3]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_221]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_221]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_221]

patduin · 2020-02-14T11:51:18Z

Can you run:
hive --hiveconf hive.metastore.uris=thrift://cg-test-mysql-0.cg-test-mysql-headless.default.svc.cluster.local:9083
Does that work? I am a bit suspicious about eh mysql in the name.

chengang2 · 2020-02-14T14:31:37Z

@patduin The hive version is incorrect. I use hive3.1.2. I tried to compile mvn clean package .but error :
[ERROR] /opt/waggle-dance-waggle-dance-parent-3.5.0/waggle-dance-core/src/main/java/com/hotels/bdp/waggledance/mapping/model/DatabaseMapping.java:[41,44] cannot find symbol
[ERROR] symbol: class Index
[ERROR] location: package org.apache.hadoop.hive.metastore.api

What should I change to make it support hive3.1.2？

patduin · 2020-02-14T14:57:20Z

Not sure I understand. Did the direct hive connection work?
WD talks thrift as long as that's compatible you can use older version of hive as client to the server.

That said we haven't moved to Hive 3 yet. I guess like you show there will be some compile errors and lots of testing to make sure we can still talk to old servers as well. This sounds like a different issue though and not your original problem.

zhangbutao · 2020-09-02T13:03:56Z

I have the same problem in hive3.1.0:
org.apache.hadoop.hive.metastore.api.NoSuchObjectException: Primary metastore does not have database @hive#default at com.hotels.bdp.waggledance.mapping.service.impl.StaticDatabaseMappingService.databaseMapping(StaticDatabaseMappingService.java:268) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at com.hotels.bdp.waggledance.mapping.service.impl.MonitoredDatabaseMappingService.databaseMapping(MonitoredDatabaseMappingService.java:48) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_tables_by_type_aroundBody716(FederatedHMSHandler.java:2131) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure717.run(FederatedHMSHandler.java:1) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.8.13.jar!/:1.8.13] at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:58) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at com.hotels.bdp.waggledance.metrics.MonitoredAspect.monitor(MonitoredAspect.java:47) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_tables_by_type_aroundBody718(FederatedHMSHandler.java:2129) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at com.hotels.bdp.waggledance.server.FederatedHMSHandler$AjcClosure719.run(FederatedHMSHandler.java:1) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjweaver-1.8.13.jar!/:1.8.13] at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:213) ~[jcabi-aspects-0.22.6.jar!/:?] at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) ~[jcabi-aspects-0.22.6.jar!/:?] at com.jcabi.aspects.aj.MethodLogger.wrapMethod(MethodLogger.java:169) ~[jcabi-aspects-0.22.6.jar!/:?] at com.hotels.bdp.waggledance.server.FederatedHMSHandler.get_tables_by_type(FederatedHMSHandler.java:2129) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112] at com.hotels.bdp.waggledance.server.ExceptionWrappingHMSHandler.invoke(ExceptionWrappingHMSHandler.java:50) ~[waggle-dance-core-3.6.1-SNAPSHOT.jar!/:?] at com.sun.proxy.$Proxy141.get_tables_by_type(Unknown Source) ~[?:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) ~[hive-standalone-metastore-3.1.0-bc3.2.0.jar!/:3.1.0-bc3.2.0] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) ~[hive-standalone-metastore-3.1.0-bc3.2.0.jar!/:3.1.0-bc3.2.0] at com.sun.proxy.$Proxy141.get_tables_by_type(Unknown Source) ~[?:?] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_tables_by_type.getResult(ThriftHiveMetastore.java:15142) ~[hive-standalone-metastore-3.1.0-bc3.2.0.jar!/:3.1.0-bc3.2.0] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_tables_by_type.getResult(ThriftHiveMetastore.java:15126) ~[hive-standalone-metastore-3.1.0-bc3.2.0.jar!/:3.1.0-bc3.2.0] at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) [libthrift-0.9.3.jar!/:0.9.3] at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) [libthrift-0.9.3.jar!/:0.9.3] at org.apache.hadoop.hive.metastore.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:48) [hive-standalone-metastore-3.1.0-bc3.2.0.jar!/:3.1.0-bc3.2.0] at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) [libthrift-0.9.3.jar!/:0.9.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]

zhangbutao · 2020-09-02T13:09:40Z

It seems that ThriftHiveMetastore would return @hive# format database name in hive3. I hope someone could give some advice to fix this. thanks !

patduin · 2020-09-02T13:41:17Z

Thanks @butaozhang for reporting. We haven't had time to look into Hive 3 support. It's in our roadmap but not actively worked on at this moment. Hopefully we'll be able to give an update or even a new version in the not too far future.

massdosage · 2020-09-03T08:55:13Z

If anyone would like to contribute support for Hive 3 we have a branch at https://github.com/HotelsDotCom/waggle-dance/tree/hive-3.x that has the Hive and Hadoop version numbers set to 3.x versions. Currently the code doesn't even compile but this could be a starting point for anyone who wants to get it working, we would happily review any pull requests against that branch that help get this all working.

massdosage · 2020-09-03T08:57:25Z

I'm going to close this issue as I've opened a new one which specifically mentions Hive 3 support - #197

patduin added the hive3 label Sep 2, 2020

massdosage closed this as completed Sep 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Primary metastore does not have database #189

Primary metastore does not have database #189

chengang2 commented Feb 13, 2020 •

edited by patduin

Loading

chengang2 commented Feb 13, 2020

massdosage commented Feb 13, 2020

patduin commented Feb 13, 2020

chengang2 commented Feb 13, 2020

chengang2 commented Feb 13, 2020

patduin commented Feb 13, 2020 •

edited

Loading

chengang2 commented Feb 14, 2020 •

edited by patduin

Loading

patduin commented Feb 14, 2020

chengang2 commented Feb 14, 2020

patduin commented Feb 14, 2020

zhangbutao commented Sep 2, 2020

zhangbutao commented Sep 2, 2020

patduin commented Sep 2, 2020

massdosage commented Sep 3, 2020

massdosage commented Sep 3, 2020

Primary metastore does not have database #189

Primary metastore does not have database #189

Comments

chengang2 commented Feb 13, 2020 • edited by patduin Loading

chengang2 commented Feb 13, 2020

massdosage commented Feb 13, 2020

patduin commented Feb 13, 2020

chengang2 commented Feb 13, 2020

chengang2 commented Feb 13, 2020

patduin commented Feb 13, 2020 • edited Loading

chengang2 commented Feb 14, 2020 • edited by patduin Loading

patduin commented Feb 14, 2020

chengang2 commented Feb 14, 2020

patduin commented Feb 14, 2020

zhangbutao commented Sep 2, 2020

zhangbutao commented Sep 2, 2020

patduin commented Sep 2, 2020

massdosage commented Sep 3, 2020

massdosage commented Sep 3, 2020

chengang2 commented Feb 13, 2020 •

edited by patduin

Loading

patduin commented Feb 13, 2020 •

edited

Loading

chengang2 commented Feb 14, 2020 •

edited by patduin

Loading