Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] 突发性报 invalid memory address or nil pointer dereference #2543

Closed
cipchk opened this issue Aug 21, 2024 · 3 comments
Closed

[BUG] 突发性报 invalid memory address or nil pointer dereference #2543

cipchk opened this issue Aug 21, 2024 · 3 comments
Assignees
@withchao
Labels
bug Categorizes issue or PR as related to a bug.

Comments

@cipchk
Copy link

cipchk commented Aug 21, 2024

OpenIM Server Version

3.8.0

Operating System and CPU Architecture

Linux (AMD)

Deployment Method

Docker Deployment

Bug Description and Steps to Reproduce

无任何预警，直接 openim-server 报出以下错误，并该服务直接当机：

	2024-08-21 09:18:58.192	WARN	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[msggateway/client.go:140]                        	readMessage                                       	{"remoteAddr": "172.30.0.1:36562", "platform": "Android", "connID": "661ccf00b8caaa109a37f39bf090e678", "operationID": "1724231852128687888", "opUserID": "10326", "messageType": "-1", "error": "read tcp 172.30.0.14:10001->172.30.0.1:36562: i/o timeout"}
	panic: runtime error: invalid memory address or nil pointer dereference
	[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x11def43]
	goroutine 522741 [running]:
	github.com/openimsdk/open-im-server/v3/pkg/common/storage/cache/redis.(*msgCache).GetMessagesBySeq.func1({0x1923da0, 0xc0017fdd10}, 0xc000ddb2e7?, {0xc000af3320?, 0x32, 0x32})
		/openim-server/pkg/common/storage/cache/redis/msg.go:167 +0x1e3
	github.com/openimsdk/open-im-server/v3/pkg/common/storage/cache/redis.ProcessKeysBySlot.func1()
		/openim-server/pkg/common/storage/cache/redis/redis_shard_manager.go:182 +0x6d
	golang.org/x/sync/errgroup.(*Group).Go.func1()
		/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78 +0x56
	created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 522739
		/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x96
	2024-08-21 09:19:09.209	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - PullMessageBySeqs     	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724231943651", "opUserID": "10366", "funcName": "/openim.msg.msg/PullMessageBySeqs", "error": "rpc error: code = Unavailable desc = error reading from server: EOF"}
	2024-08-21 09:19:23.420	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - PullMessageBySeqs     	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724231963195", "opUserID": "10366", "funcName": "/openim.msg.msg/PullMessageBySeqs", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\""}
	2024-08-21 09:19:51.092	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724231993454859285", "opUserID": "10366", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:19:51.109	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724231990506", "opUserID": "10366", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:19:51.887	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724231993727428437", "opUserID": "10366", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:19:51.898	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724231990711", "opUserID": "10366", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:19:55.093	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724231997926597916", "opUserID": "10366", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:19:55.104	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724231994189", "opUserID": "10366", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:21:07.190	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - SearchMessage         	{"platform": "Admin", "operationID": "063397fa-1861-4117-b8b3-4250f6da776e", "opUserID": "imAdmin", "funcName": "/openim.msg.msg/SearchMessage", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:21:11.763	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - SearchMessage         	{"platform": "Admin", "operationID": "685dbe3b-1ee5-42c7-a43f-e839d54b57a0", "opUserID": "imAdmin", "funcName": "/openim.msg.msg/SearchMessage", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:21:30.719	WARN	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[msggateway/client.go:140]                        	readMessage                                       	{"remoteAddr": "172.30.0.1:53304", "platform": "Android", "connID": "95df5691ae2661fc383c21a8c8fc1926", "operationID": "1724231550755696624", "opUserID": "10274", "messageType": "-1", "error": "read tcp 172.30.0.14:10001->172.30.0.1:53304: i/o timeout"}
	2024-08-21 09:21:53.188	WARN	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[msggateway/client.go:140]                        	readMessage                                       	{"remoteAddr": "172.30.0.1:41364", "platform": "Android", "connID": "453909c5e790092fc8eeb1d23033f71a", "operationID": "1724231504312537961", "opUserID": "10300", "messageType": "-1", "error": "read tcp 172.30.0.14:10001->172.30.0.1:41364: i/o timeout"}
	2024-08-21 09:22:05.484	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "ddfcdf251ebc09ae48809e1b3fcf2566", "operationID": "1724227857243", "opUserID": "10346", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:22:06.748	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724227857243", "opUserID": "10346", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:22:34.215	WARN	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[msggateway/client.go:140]                        	readMessage                                       	{"remoteAddr": "172.30.0.1:54520", "platform": "Android", "connID": "deb7a9b20e089938494065a68cd20cb9", "operationID": "1724231941476187150", "opUserID": "10326", "messageType": "-1", "error": "read tcp 172.30.0.14:10001->172.30.0.1:54520: i/o timeout"}
	2024-08-21 09:22:37.529	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "8433901f994d701f835435752b54cfeb", "operationID": "1724219829473", "opUserID": "10326", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:22:39.035	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724219829473", "opUserID": "10326", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:22:50.196	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724232167318", "opUserID": "10321", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:22:50.199	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "dd2ea2d4fd9de572e366f4aa5b00cd4f", "operationID": "1724232167318", "opUserID": "10321", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:07.549	WARN	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[msggateway/client.go:140]                        	readMessage                                       	{"remoteAddr": "172.30.0.1:52492", "platform": "Android", "connID": "8433901f994d701f835435752b54cfeb", "operationID": "1724232154427024590", "opUserID": "10326", "messageType": "-1", "error": "read tcp 172.30.0.14:10001->172.30.0.1:52492: i/o timeout"}
	2024-08-21 09:23:09.812	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "6fa77d173bc130888f6bffcd6753d198", "operationID": "1724219829473", "opUserID": "10326", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:09.826	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724219829473", "opUserID": "10326", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:20.445	WARN	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[msggateway/client.go:140]                        	readMessage                                       	{"remoteAddr": "172.30.0.1:42526", "platform": "Android", "connID": "ddfcdf251ebc09ae48809e1b3fcf2566", "operationID": "1724232125908237498", "opUserID": "10346", "messageType": "-1", "error": "read tcp 172.30.0.14:10001->172.30.0.1:42526: i/o timeout"}
	2024-08-21 09:23:21.858	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724232203475106492", "opUserID": "10366", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:21.936	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724232201274", "opUserID": "10366", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:22.517	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "8a29d4eb665146fe897ebb877e7f3264", "operationID": "1724227857243", "opUserID": "10346", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:22.673	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724232206227597910", "opUserID": "10366", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:22.700	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724232201444", "opUserID": "10366", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:23.610	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724227857243", "opUserID": "10346", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:29.156	ERROR	[PID:1621]     	openim-api               	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetConversationsHasReadAndMaxSeq	{"platform": "Android", "operationID": "1724232208244", "opUserID": "10366", "funcName": "/openim.msg.msg/GetConversationsHasReadAndMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:29.174	ERROR	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[mw/rpc_client_interceptor.go:50]                 	RPC Client Response Error - GetMaxSeq             	{"platform": "Android", "connID": "bb022eb62ede520229a4522c26cb89aa", "operationID": "1724232209574825373", "opUserID": "10366", "funcName": "/openim.msg.msg/GetMaxSeq", "error": "rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.0.1.150:10130: connect: connection refused\"; last resolver error: produced zero addresses"}
	2024-08-21 09:23:52.518	WARN	[PID:1598]     	openim-msggateway        	[version:3.8.0]  	[msggateway/client.go:140]

重启服务后，恢复正常。

Screenshots Link

No response
@cipchk cipchk added the bug Categorizes issue or PR as related to a bug. label Aug 21, 2024
@FGadvancer
Copy link
Member

Can you provide information use "docker images" for us to troubleshoot, thanky!!!

@cipchk
Copy link
Author

cipchk commented Aug 22, 2024 

openim/openim-web-front                              release-v3.8.0                 b08f406b8c68   2 weeks ago         85.7MB
openim/openim-chat                                   release-v1.8                   5c3f739cfc42   2 weeks ago         465MB
openim/openim-server                                 release-v3.8                   07c845b2afda   2 weeks ago         985MB
openim/openim-admin-front                            release-v1.8.0                 2df59c51dcbf   2 weeks ago         48.1MB
grafana/grafana                                      11.0.1                         97bf66938f26   8 weeks ago         445MB
redis                                                latest                         9c893be668ac   3 months ago        116MB
prom/prometheus                                      v2.51.2                        051cb67876a6   4 months ago        262MB
quay.io/coreos/etcd                                  v3.5.13                        bef2e3b99162   4 months ago        57.8MB
prom/alertmanager                                    v0.27.0                        11f11916f8cd   5 months ago        70.2MB
minio/minio                                          RELEASE.2024-01-11T07-46-16Z   f5d82361e10f   7 months ago        151MB
bitnami/kafka                                        3.5.1                          f79889e4e4af   8 months ago        558MB
prom/node-exporter                                   v1.7.0                         72c9c2088986   9 months ago        22.7MB
mongo                                                6.0.2                          b70536aeb250   22 months ago       695MB
redis                                                7.0.0                          53aa81e8adfa   2 years ago         117MB

@withchao withchao mentioned this issue Sep 29, 2024
Merged
@withchao
Copy link
Contributor

Does this problem occur again? Judging from the code, this problem should not occur.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@withchao withchao

Labels
bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cipchk @FGadvancer @withchao