Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

运行一夜后核心无法接受任何连接，错误：too many open files #4194

Closed
4 tasks done
jpsn123 opened this issue Dec 22, 2024 · 34 comments
Closed
4 tasks done

运行一夜后核心无法接受任何连接，错误：too many open files #4194

jpsn123 opened this issue Dec 22, 2024 · 34 comments

Comments

@jpsn123
Copy link

jpsn123 commented Dec 22, 2024 •
edited
Loading

完整性要求

  • 我保证阅读了文档，了解所有我编写的配置文件项的含义，而不是大量堆砌看似有用的选项或默认值。
  • 我提供了完整的配置文件和日志，而不是出于自己的判断只给出截取的部分。
  • 我搜索了 issues, 没有发现已提出的类似问题。
  • 问题在 Release 最新的版本上可以成功复现

描述

背景介绍：
我是使用openwrt安装passwall2插件来间接使用xray-core，passwall2是配置的透明代理，提供给50人团队日常使用，以前使用一直很正常，就是代理相比clash有点慢。最近更新了openwrt，passwall2也是用最新版，xray-core核心也升级到了24.11.30版本。新的环境配置好后一切正常，而且比升级前快不少，除了有时解析国内网站DNS慢（应该是把国内站点的域名路由到国外了解析了）。

问题出现
刚刚启动xray是没有任何问题的，但是过一夜后，xray无法接受任何路由到1041端口的流量，日志提示[Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files。
此时xray核心无法正常关闭，重启命令/etc/init.d/passwall2 restart会一直被阻塞住。
使用命令netstat -an会打印出超多udp连接，我截取一段：

udp        0      0 120.50.73.220:41001     0.0.0.0:*
udp        0      0 24.135.132.230:59434    0.0.0.0:*
udp        0      0 90.163.53.175:18474     0.0.0.0:*
udp        0      0 81.171.17.82:53290      0.0.0.0:*
udp        0      0 1.164.227.248:2090      0.0.0.0:*
udp        0      0 36.156.195.68:8234      0.0.0.0:*
udp        0      0 64.99.218.98:63530      0.0.0.0:*
udp        0      0 112.24.204.35:26666     0.0.0.0:*
udp        0      0 58.218.213.211:61482    0.0.0.0:*
udp        0      0 120.238.235.54:22570    0.0.0.0:*
udp        0      0 190.120.255.28:51242    0.0.0.0:*
udp        0      0 120.153.93.92:45098     0.0.0.0:*
udp        0      0 36.250.241.154:22570    0.0.0.0:*
udp        0      0 125.77.27.52:36906      0.0.0.0:*
udp        0      0 1.175.37.198:10282      0.0.0.0:*
udp        0      0 176.109.255.122:59434   0.0.0.0:*
udp        0      0 62.73.72.251:26666      0.0.0.0:*
udp        0      0 24.90.52.72:51242       0.0.0.0:*
udp        0      0 125.77.27.33:28714      0.0.0.0:*
udp        0      0 38.89.70.185:61482      0.0.0.0:*
udp        0      0 175.194.251.123:8234    0.0.0.0:*
udp        0      0 178.234.44.238:14378    0.0.0.0:*
udp        0      0 89.149.202.1:55338      0.0.0.0:*
udp        0      0 112.46.3.47:24618       0.0.0.0:*
udp        0      0 188.32.254.182:53290    0.0.0.0:*

这些udp连接没有被正确释放，所以go的socket无法接受新的连接，网上有提升limit的方式，但我感觉这个问题不是达到上限，而是连接泄露。
我开始我以为是openwrt问题，我尝试过切换到以前的openwrt稳定版本，5.15内核，使用最新的xray，问题仍然存在。passwall2只是一个配置生成器，生成的配置文件我检查了也没有问题，应该不会是passwall2的问题，所以我想很可能是xray问题。

恢复方法
只能强制kill掉xray-core进程，之后一切正常，一夜之后又是同样的错误。

重现方式

bug比较难以重现，内网机器要多（其实就是上网用户很多），然后等待一段时间，必然出现

客户端配置

配置文件已去密，由passwall2最新版生成，配置文件没有什么问题，因为一开始是可以正常工作的


{
  "outbounds": [
    {
      "streamSettings": {
        "sockopt": {
          "mark": 255
        }
      },
      "settings": {
        "domainStrategy": "UseIP"
      },
      "protocol": "freedom",
      "tag": "direct"
    },
    {
      "mux": {
        "enabled": false
      },
      "protocol": "trojan",
      "settings": {
        "servers": [
          {
            "password": "xxxx",
            "port": 8443,
            "address": "xxxx"
          }
        ]
      },
      "streamSettings": {
        "network": "raw",
        "tlsSettings": {
          "serverName": "xxxx",
          "allowInsecure": true
        },
        "sockopt": {
          "mark": 255
        },
        "rawSettings": {
          "header": {
            "type": "none"
          }
        },
        "security": "tls"
      },
      "tag": "ProxyGame:DMIT-US-AI-NF-PRO-gw-100-trojan"
    },
    {
      "mux": {
        "enabled": false
      },
      "protocol": "trojan",
      "streamSettings": {
        "network": "raw",
        "tlsSettings": {
          "allowInsecure": true,
          "serverName": "xxxx"
        },
        "sockopt": {
          "mark": 255
        },
        "rawSettings": {
          "header": {
            "type": "none"
          }
        },
        "security": "tls"
      },
      "settings": {
        "servers": [
          {
            "password": "xxxx",
            "port": 8443,
            "address": "xxxx"
          }
        ]
      },
      "tag": "GooglePlay:DMIT-US-AI-NF-PRO-gw-100-trojan"
    },
    {
      "mux": {
        "enabled": false
      },
      "protocol": "trojan",
      "streamSettings": {
        "network": "raw",
        "tlsSettings": {
          "allowInsecure": true,
          "serverName": "xxxx"
        },
        "sockopt": {
          "mark": 255
        },
        "rawSettings": {
          "header": {
            "type": "none"
          }
        },
        "security": "tls"
      },
      "settings": {
        "servers": [
          {
            "password": "xxxx",
            "port": 8443,
            "address": "xxxx"
          }
        ]
      },
      "tag": "Netflix:DMIT-US-AI-NF-PRO-gw-100-trojan"
    },
    {
      "mux": {
        "enabled": false
      },
      "protocol": "trojan",
      "streamSettings": {
        "network": "raw",
        "tlsSettings": {
          "allowInsecure": true,
          "serverName": "xxxx"
        },
        "sockopt": {
          "mark": 255
        },
        "rawSettings": {
          "header": {
            "type": "none"
          }
        },
        "security": "tls"
      },
      "settings": {
        "servers": [
          {
            "password": "xxxx",
            "port": 8443,
            "address": "xxxx"
          }
        ]
      },
      "tag": "OpenAI:DMIT-US-AI-NF-PRO-gw-100-trojan"
    },
    {
      "mux": {
        "enabled": false
      },
      "protocol": "trojan",
      "streamSettings": {
        "network": "raw",
        "tlsSettings": {
          "allowInsecure": true,
          "serverName": "xxxx"
        },
        "sockopt": {
          "mark": 255
        },
        "rawSettings": {
          "header": {
            "type": "none"
          }
        },
        "security": "tls"
      },
      "settings": {
        "servers": [
          {
            "password": "xxxx",
            "port": 8443,
            "address": "xxxx"
          }
        ]
      },
      "tag": "Docker:DMIT-US-AI-NF-PRO-gw-100-trojan"
    },
    {
      "mux": {
        "enabled": false
      },
      "protocol": "trojan",
      "streamSettings": {
        "network": "raw",
        "tlsSettings": {
          "allowInsecure": true,
          "serverName": "xxxx"
        },
        "sockopt": {
          "mark": 255
        },
        "rawSettings": {
          "header": {
            "type": "none"
          }
        },
        "security": "tls"
      },
      "settings": {
        "servers": [
          {
            "password": "xxxx",
            "port": 8443,
            "address": "xxxx"
          }
        ]
      },
      "tag": "Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan"
    },
    {
      "proxySettings": {
        "tag": "direct"
      },
      "settings": {
        "port": 15354,
        "network": "udp",
        "address": "127.0.0.1",
        "blockTypes": [
          65
        ],
        "nonIPQuery": "skip"
      },
      "protocol": "dns",
      "tag": "dns-out"
    },
    {
      "protocol": "blackhole",
      "tag": "blackhole"
    }
  ],
  "log": {
    "loglevel": "warning"
  },
  "dns": {
    "disableFallback": true,
    "queryStrategy": "UseIP",
    "disableFallbackIfMatch": true,
    "servers": [
      {
        "queryStrategy": "UseIP",
        "port": 15354,
        "_flag": "default",
        "address": "127.0.0.1"
      },
      {
        "port": 15354,
        "_flag": "direct",
        "address": "127.0.0.1",
        "queryStrategy": "UseIP",
        "domains": [
          "full:xxxx",
          "full:xxxx"
        ]
      },
      {
        "domains": [
          "geosite:category-ads-all"
        ],
        "_flag": "AD",
        "address": "fakedns"
      },
      {
        "port": 15354,
        "_flag": "DirectGame",
        "address": "127.0.0.1",
        "queryStrategy": "UseIP",
        "domains": [
          "api.steampowered.com",
          "regexp:\\.cm.steampowered.com$",
          "regexp:\\.steamserver.net$",
          "geosite:category-games@cn"
        ]
      },
      {
        "domains": [
          "geosite:category-games",
          "domain:store.steampowered.com"
        ],
        "_flag": "ProxyGame",
        "address": "fakedns"
      },
      {
        "port": 15354,
        "_flag": "Direct",
        "address": "127.0.0.1",
        "queryStrategy": "UseIP",
        "domains": [
          "apple.com",
          "microsoft.com",
          "dyndns.com",
          "steamcontent.com",
          "dl.steam.clngaa.com",
          "dl.steam.ksyna.com",
          "st.dl.bscstorage.net",
          "st.dl.eccdnx.com",
          "st.dl.pinyuncloud.com",
          "cdn.mileweb.cs.steampowered.com.8686c.com",
          "cdn-ws.content.steamchina.com",
          "cdn-qc.content.steamchina.com",
          "cdn-ali.content.steamchina.com",
          "epicgames-download1-1251447533.file.myqcloud.com"
        ]
      },
      {
        "port": 15354,
        "_flag": "China",
        "address": "127.0.0.1",
        "queryStrategy": "UseIP",
        "domains": [
          "geosite:cn"
        ]
      },
      {
        "domains": [
          "domain:googleapis.cn",
          "domain:googleapis.com",
          "domain:xn--ngstr-lra8j.com"
        ],
        "_flag": "GooglePlay",
        "address": "fakedns"
      },
      {
        "domains": [
          "geosite:netflix"
        ],
        "_flag": "Netflix",
        "address": "fakedns"
      },
      {
        "domains": [
          "geosite:openai"
        ],
        "_flag": "OpenAI",
        "address": "fakedns"
      },
      {
        "domains": [
          "geosite:docker",
          "gcr.io"
        ],
        "_flag": "Docker",
        "address": "fakedns"
      },
      {
        "domains": [
          "geosite:geolocation-!cn",
          "v2ex.com",
          "nextcloud.com",
          "truenas.com",
          "truenas.net",
          "unraid.net",
          "alpinelinux.org",
          "freertos.org",
          "openstreetmap.org",
          "esphome.io",
          "jaggaer.com",
          "onlyoffice.com",
          "proxmox.com",
          "docs.fortinet.com",
          "emby.media",
          "k8s.io",
          "ledger.com",
          "dmit.io"
        ],
        "_flag": "Proxy",
        "address": "fakedns"
      }
    ],
    "disableCache": false,
    "tag": "dns-in1"
  },
  "routing": {
    "rules": [
      {
        "inboundTag": [
          "dns-in"
        ],
        "outboundTag": "dns-out"
      },
      {
        "outboundTag": "direct",
        "port": 15354,
        "network": "udp",
        "ip": [
          "127.0.0.1"
        ]
      },
      {
        "domains": [
          "geosite:category-ads-all"
        ],
        "ruleTag": "AD Domains",
        "network": "tcp,udp",
        "outboundTag": "blackhole"
      },
      {
        "network": "tcp,udp",
        "outboundTag": "direct",
        "ruleTag": "BT",
        "protocol": [
          "bittorrent"
        ]
      },
      {
        "domains": [
          "api.steampowered.com",
          "regexp:\\.cm.steampowered.com$",
          "regexp:\\.steamserver.net$",
          "geosite:category-games@cn"
        ],
        "ruleTag": "DirectGame Domains",
        "network": "tcp,udp",
        "outboundTag": "direct"
      },
      {
        "ip": [
          "103.10.124.0\/24",
          "103.10.125.0\/24",
          "103.28.54.0\/24",
          "146.66.152.0\/24",
          "146.66.155.0\/24",
          "153.254.86.0\/24",
          "155.133.224.0\/23",
          "155.133.226.0\/24",
          "155.133.227.0\/24",
          "155.133.230.0\/24",
          "155.133.232.0\/24",
          "155.133.233.0\/24",
          "155.133.234.0\/24",
          "155.133.236.0\/23",
          "155.133.238.0\/24",
          "155.133.239.0\/24",
          "155.133.240.0\/23",
          "155.133.245.0\/24",
          "155.133.246.0\/24",
          "155.133.248.0\/24",
          "155.133.249.0\/24",
          "155.133.250.0\/24",
          "155.133.251.0\/24",
          "155.133.252.0\/24",
          "155.133.253.0\/24",
          "155.133.254.0\/24",
          "155.133.255.0\/24",
          "162.254.192.0\/24",
          "162.254.193.0\/24",
          "162.254.194.0\/23",
          "162.254.195.0\/24",
          "162.254.196.0\/24",
          "162.254.197.0\/24",
          "162.254.198.0\/24",
          "162.254.199.0\/24",
          "185.25.182.0\/24",
          "185.25.183.0\/24",
          "190.217.33.0\/24",
          "192.69.96.0\/22",
          "205.185.194.0\/24",
          "205.196.6.0\/24",
          "208.64.200.0\/24",
          "208.64.201.0\/24",
          "208.64.202.0\/24",
          "208.64.203.0\/24",
          "208.78.164.0\/22"
        ],
        "ruleTag": "DirectGame IP",
        "network": "tcp,udp",
        "outboundTag": "direct"
      },
      {
        "domains": [
          "geosite:category-games",
          "domain:store.steampowered.com"
        ],
        "ruleTag": "ProxyGame Domains",
        "network": "tcp,udp",
        "outboundTag": "ProxyGame:DMIT-US-AI-NF-PRO-gw-100-trojan"
      },
      {
        "domains": [
          "apple.com",
          "microsoft.com",
          "dyndns.com",
          "steamcontent.com",
          "dl.steam.clngaa.com",
          "dl.steam.ksyna.com",
          "st.dl.bscstorage.net",
          "st.dl.eccdnx.com",
          "st.dl.pinyuncloud.com",
          "cdn.mileweb.cs.steampowered.com.8686c.com",
          "cdn-ws.content.steamchina.com",
          "cdn-qc.content.steamchina.com",
          "cdn-ali.content.steamchina.com",
          "epicgames-download1-1251447533.file.myqcloud.com"
        ],
        "ruleTag": "Direct Domains",
        "network": "tcp,udp",
        "outboundTag": "direct"
      },
      {
        "ip": [
          "geoip:private",
          "114.114.114.114",
          "114.114.115.115",
          "223.5.5.5",
          "223.6.6.6",
          "119.29.29.29",
          "180.76.76.76"
        ],
        "ruleTag": "Direct IP",
        "network": "tcp,udp",
        "outboundTag": "direct"
      },
      {
        "domains": [
          "geosite:cn"
        ],
        "ruleTag": "China Domains",
        "network": "tcp,udp",
        "outboundTag": "direct"
      },
      {
        "ip": [
          "geoip:cn"
        ],
        "ruleTag": "China IP",
        "network": "tcp,udp",
        "outboundTag": "direct"
      },
      {
        "network": "udp",
        "outboundTag": "blackhole",
        "port": "443",
        "ruleTag": "QUIC"
      },
      {
        "domains": [
          "domain:googleapis.cn",
          "domain:googleapis.com",
          "domain:xn--ngstr-lra8j.com"
        ],
        "ruleTag": "GooglePlay Domains",
        "network": "tcp,udp",
        "outboundTag": "GooglePlay:DMIT-US-AI-NF-PRO-gw-100-trojan"
      },
      {
        "domains": [
          "geosite:netflix"
        ],
        "ruleTag": "Netflix Domains",
        "network": "tcp,udp",
        "outboundTag": "Netflix:DMIT-US-AI-NF-PRO-gw-100-trojan"
      },
      {
        "domains": [
          "geosite:openai"
        ],
        "ruleTag": "OpenAI Domains",
        "network": "tcp,udp",
        "outboundTag": "OpenAI:DMIT-US-AI-NF-PRO-gw-100-trojan"
      },
      {
        "domains": [
          "geosite:docker",
          "gcr.io"
        ],
        "ruleTag": "Docker Domains",
        "network": "tcp,udp",
        "outboundTag": "Docker:DMIT-US-AI-NF-PRO-gw-100-trojan"
      },
      {
        "domains": [
          "geosite:geolocation-!cn",
          "v2ex.com",
          "nextcloud.com",
          "truenas.com",
          "truenas.net",
          "unraid.net",
          "alpinelinux.org",
          "freertos.org",
          "openstreetmap.org",
          "esphome.io",
          "jaggaer.com",
          "onlyoffice.com",
          "proxmox.com",
          "docs.fortinet.com",
          "emby.media",
          "k8s.io",
          "ledger.com",
          "dmit.io"
        ],
        "ruleTag": "Proxy Domains",
        "network": "tcp,udp",
        "outboundTag": "Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan"
      },
      {
        "ip": [
          "149.154.160.0\/20",
          "91.108.4.0\/22",
          "91.108.56.0\/24",
          "109.239.140.0\/24",
          "67.198.55.0\/24",
          "8.8.4.4",
          "8.8.8.8",
          "208.67.222.222",
          "208.67.220.220",
          "1.1.1.1",
          "1.1.1.2",
          "1.0.0.1",
          "9.9.9.9",
          "149.112.112.112",
          "2001:67c:4e8::\/48",
          "2001:b28:f23c::\/48",
          "2001:b28:f23d::\/48",
          "2001:b28:f23f::\/48",
          "2001:b28:f242::\/48",
          "2001:4860:4860::8888",
          "2001:4860:4860::8844",
          "2606:4700:4700::1111",
          "2606:4700:4700::1001"
        ],
        "ruleTag": "Proxy IP",
        "network": "tcp,udp",
        "outboundTag": "Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan"
      }
    ],
    "domainStrategy": "IPOnDemand",
    "domainMatcher": "hybrid"
  },
  "policy": {
    "levels": {
      "0": {
        "statsUserUplink": false,
        "statsUserDownlink": false
      }
    }
  },
  "fakedns": [
    {
      "ipPool": "198.18.0.0\/16",
      "poolSize": 65535
    }
  ],
  "inbounds": [
    {
      "port": 1070,
      "protocol": "socks",
      "sniffing": {
        "enabled": true,
        "routeOnly": true,
        "destOverride": [
          "http",
          "tls",
          "quic"
        ]
      },
      "settings": {
        "udp": true,
        "auth": "noauth"
      },
      "listen": "127.0.0.1",
      "tag": "socks-in"
    },
    {
      "port": 1041,
      "protocol": "dokodemo-door",
      "streamSettings": {
        "sockopt": {
          "tproxy": "redirect"
        }
      },
      "sniffing": {
        "enabled": true,
        "routeOnly": true,
        "destOverride": [
          "http",
          "tls",
          "quic",
          "fakedns"
        ],
        "metadataOnly": false
      },
      "settings": {
        "network": "tcp",
        "followRedirect": true
      },
      "tag": "tcp_redir"
    },
    {
      "port": 1041,
      "protocol": "dokodemo-door",
      "streamSettings": {
        "sockopt": {
          "tproxy": "tproxy"
        }
      },
      "sniffing": {
        "enabled": true,
        "routeOnly": true,
        "destOverride": [
          "http",
          "tls",
          "quic",
          "fakedns"
        ],
        "metadataOnly": false
      },
      "settings": {
        "network": "udp",
        "followRedirect": true
      },
      "tag": "udp_redir"
    },
    {
      "port": 15353,
      "protocol": "dokodemo-door",
      "settings": {
        "network": "tcp,udp",
        "address": "0.0.0.0"
      },
      "tag": "dns-in",
      "listen": "127.0.0.1"
    }
  ]
}

服务端配置

无关服务端

客户端日志

截取部分日志


2024/12/22 13:38:46 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:46 from 127.0.0.1:46688 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:47 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:47 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:47 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:47 from 127.0.0.1:45541 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:47 from 10.12.0.7:59979 accepted udp:115.231.229.43:443 [udp_redir -> direct]
2024/12/22 13:38:48 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:48 from 10.12.0.7:59979 accepted udp:115.231.229.43:443 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59980 accepted udp:115.231.229.43:8000 [udp_redir -> direct]
2024/12/22 13:38:48 from 127.0.0.1:41970 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:48 from 10.12.0.7:59980 accepted udp:115.231.229.43:8000 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59979 accepted udp:115.231.229.43:443 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59981 accepted udp:115.231.229.156:443 [udp_redir -> direct]
2024/12/22 13:38:48 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:48 from 10.12.0.7:59981 accepted udp:115.231.229.156:443 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59980 accepted udp:115.231.229.43:8000 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59982 accepted udp:115.231.229.156:8000 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59982 accepted udp:115.231.229.156:8000 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59981 accepted udp:115.231.229.156:443 [udp_redir -> direct]
2024/12/22 13:38:48 from 10.12.0.7:59979 accepted udp:115.231.229.43:443 [udp_redir -> direct]
2024/12/22 13:38:49 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:49 from 127.0.0.1:34735 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:49 from 10.12.0.7:59982 accepted udp:115.231.229.156:8000 [udp_redir -> direct]
2024/12/22 13:38:49 from 10.12.0.7:59980 accepted udp:115.231.229.43:8000 [udp_redir -> direct]
2024/12/22 13:38:49 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:49 from 10.12.0.7:59981 accepted udp:115.231.229.156:443 [udp_redir -> direct]
2024/12/22 13:38:49 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:49 from 10.12.0.7:59982 accepted udp:115.231.229.156:8000 [udp_redir -> direct]
2024/12/22 13:38:49 from 127.0.0.1:48068 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:50 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:50 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:50 from 127.0.0.1:37687 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:50 from 127.0.0.1:47790 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:50 from 127.0.0.1:39873 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:50 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:51 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:51 from 127.0.0.1:35394 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:51 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:51 from DNS accepted udp:127.0.0.1:15354 [dns-in1 -> direct]
2024/12/22 13:38:51 from 127.0.0.1:35152 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:51 from 127.0.0.1:50230 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:51 from 127.0.0.1:46146 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:52 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:52 from 127.0.0.1:49138 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:52 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:52 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:52 from 127.0.0.1:59863 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:52 from 127.0.0.1:36784 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:52 from 127.0.0.1:52640 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:52 from 127.0.0.1:34848 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:52 from 127.0.0.1:49604 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:52 from 127.0.0.1:35967 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:53 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:53 from 127.0.0.1:45208 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:53 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:53 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:53 from 127.0.0.1:55406 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:53 from 127.0.0.1:60076 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:53 from 127.0.0.1:38656 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:54 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:54 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:54 from 127.0.0.1:47046 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:54 from 127.0.0.1:47397 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:55 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:55 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:55 from 127.0.0.1:57206 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:55 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:55 from 10.11.0.109:57945 accepted tcp:198.18.199.123:443 [tcp_redir -> direct]
2024/12/22 13:38:55 from DNS accepted udp:127.0.0.1:15354 [dns-in1 -> direct]
2024/12/22 13:38:55 from 127.0.0.1:54962 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:55 from 127.0.0.1:53883 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:56 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:56 from 127.0.0.1:44819 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:56 from 127.0.0.1:33322 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:56 from 127.0.0.1:59626 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:56 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:57 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:57 from 127.0.0.1:47456 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:57 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:57 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:58 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:58 from 127.0.0.1:43883 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:58 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]
2024/12/22 13:38:58 from 127.0.0.1:33945 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:58 from 127.0.0.1:36649 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:58 from 10.12.0.7:64197 accepted udp:58.216.4.101:443 [udp_redir -> direct]
2024/12/22 13:38:58 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:58 from 10.12.0.7:64197 accepted udp:58.216.4.101:443 [udp_redir -> direct]
2024/12/22 13:38:58 from 127.0.0.1:48780 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:58 from 10.12.0.7:64198 accepted udp:58.216.102.69:443 [udp_redir -> direct]
2024/12/22 13:38:58 from 127.0.0.1:42363 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:58 from 10.12.0.7:64198 accepted udp:58.216.102.69:443 [udp_redir -> direct]
2024/12/22 13:38:58 from 10.12.0.7:64197 accepted udp:58.216.4.101:443 [udp_redir -> direct]
2024/12/22 13:38:58 from 10.12.0.7:64198 accepted udp:58.216.102.69:443 [udp_redir -> direct]
2024/12/22 13:38:58 from 10.12.0.7:64197 accepted udp:58.216.4.101:443 [udp_redir -> direct]
2024/12/22 13:38:59 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:59 from 127.0.0.1:47017 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:38:59 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:38:59 from DNS accepted udp:127.0.0.1:15354 [dns-in1 -> direct]
2024/12/22 13:38:59 from 127.0.0.1:59628 accepted udp:0.0.0.0:0 [dns-in -> dns-out]
2024/12/22 13:39:00 [Warning] transport/internet/tcp: failed to accepted raw connections > accept tcp [::]:1041: accept4: too many open files
2024/12/22 13:39:00 from 10.11.0.12:48753 accepted udp:198.18.194.48:21116 [udp_redir -> Proxy:DMIT-US-AI-NF-PRO-gw-100-trojan]

服务端日志

无关服务端
@RPRX
Copy link
Member

RPRX commented Dec 22, 2024

按理来说五分钟不活动会自动清掉

@jpsn123
Copy link
Author

jpsn123 commented Dec 22, 2024

自动清除是内核做的还是xray-core做的

@jpsn123
Copy link
Author

jpsn123 commented Dec 22, 2024 •
edited
Loading

刚刚发现这个问题两年前就有人遇到过了，#2073 #650

共性就是网络环境中UDP连接较多，且UDP的fd没有被自动清除，我这里使用的人多，无论TCP还是UDP，每天的流量都有一两个TB，UDP连接触发限制也就一两天的事，我以前没有遇到过估计是每天自动更新订阅的时候会重启xray-core，现在是周订阅频率。

所以我认为这个应该是xray-core一直存在的一个隐形BUG，请求开发者重视，感谢了@RPRX

@xqzr
Copy link
Contributor

xqzr commented Dec 22, 2024

自动清除是内核做的还是xray-core做的

*ray
https://xtls.github.io/config/policy.html#policyobject

@jpsn123
Copy link
Author

jpsn123 commented Dec 22, 2024

@xqzr 感谢，我看了，用的都是默认值，300s的超时感觉只对TCP生效了

@RPRX
Copy link
Member

RPRX commented Dec 26, 2024

看了下代码好像是有这个 bug，晚点我修下

@LearZhou
Copy link

是不是等這個修好才會發新版？

@RPRX
Copy link
Member

RPRX commented Dec 26, 2024

是不是等這個修好才會發新版？

想等 XUDP 文章发出后再发新版，之前说圣诞发文，结果是我太天真了，最近天天在外面玩

@RPRX RPRX closed this as completed in 2782f13 Dec 26, 2024
@RPRX
Copy link
Member

RPRX commented Dec 26, 2024 •
edited
Loading

"defer pw.Close()" 用于清理 fake UDP，之前是写在主函数 Process() 里的，但由于末尾 task.Run() 的特性，可能 Process() 都 return 了但 responseDone() 还在执行，导致提前清理了 fake UDP、后面的没清理

2782f13 把 "defer pw.Close()" 移到 responseDone() 里了，试试 https://github.com/XTLS/Xray-core/actions/runs/12504877161

还是因为没怎么看过 *ray 的基础架构，明年有望重构一下

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024

"defer pw.Close()" 用于清理 fake UDP，之前是写在主函数 Process() 里的，但由于末尾 task.Run() 的特性，可能 Process() 都 return 了但 responseDone() 还在执行，导致提前清理了 fake UDP、后面的没清理

2782f13 把 "defer pw.Close()" 移到 responseDone() 里了，试试 https://github.com/XTLS/Xray-core/actions/runs/12504877161

还是因为没怎么看过 *ray 的基础架构，明年有望重构一下

辛苦了，我测验一下，尽快给出结果

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024

@RPRX 测试了一下，问题应该没有被修复，目前udp连接已经上升到1w+，即使我把LAN区域设备断网（就等于拔网线），等待十几分钟后，udp连接也没有减少。

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024

@Fangliding 问题任然存在，根[RPRX]的提交一样，期间UDP有少量减少，但是整体还是飞速上升，5分钟2000个连接，断网十分钟后，UDP连接任然不减少（还增长几十个）。

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024 •
edited
Loading

切换clash 20分钟后，网关的网络连接情况：

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
udp        0      0 0.0.0.0:52864           0.0.0.0:*                           2665/dnsmasq
udp        0      0 198.18.0.1:53           0.0.0.0:*                           2665/dnsmasq
udp        0      0 127.0.0.1:53            0.0.0.0:*                           2665/dnsmasq
udp        0      0 192.168.100.1:53        0.0.0.0:*                           2665/dnsmasq
udp        0      0 218.94.242.236:53       0.0.0.0:*                           2665/dnsmasq
udp        0      0 0.0.0.0:67              0.0.0.0:*                           2665/dnsmasq
udp        0      0 :::41481                :::*                                2480/clash
udp        0      0 :::546                  :::*                                11818/odhcp6c
udp        0      0 :::546                  :::*                                11364/odhcp6c
udp        0      0 :::547                  :::*                                2481/odhcpd
udp        0      0 :::43711                :::*                                2480/clash
udp        0      0 :::33647                :::*                                2480/clash
udp        0      0 :::33667                :::*                                2480/clash
udp        0      0 :::56394                :::*                                2480/clash
udp        0      0 :::58513                :::*                                2480/clash
udp        0      0 :::54438                :::*                                2480/clash
udp        0      0 :::48329                :::*                                2480/clash
udp        0      0 :::54574                :::*                                2480/clash
udp        0      0 :::58971                :::*                                2480/clash
udp        0      0 :::7874                 :::*                                2480/clash
udp        0      0 :::40651                :::*                                2480/clash
udp        0      0 :::7891                 :::*                                2480/clash
udp        0      0 :::7892                 :::*                                2480/clash
udp        0      0 :::7893                 :::*                                2480/clash
udp        0      0 :::7895                 :::*                                2480/clash
udp        0      0 :::55204                :::*                                2480/clash
udp        0      0 fe80::acaa:9fff:b49d:6fc6:53 :::*                                2665/dnsmasq
udp        0      0 ::1:53                  :::*                                2665/dnsmasq
udp        0      0 fe80::6a27:19ff:fea5:5eb2:53 :::*                                2665/dnsmasq
udp        0      0 240e:3a4:4c7d:8b00::1:53 :::*                                2665/dnsmasq
udp        0      0 fde8:e2a8:319::1:53     :::*                                2665/dnsmasq
udp        0      0 fe80::6827:19ff:fea5:5eb2:53 :::*                                2665/dnsmasq
udp        0      0 240e:3a0:4c2e:c074:39e8:1af0:77a7:c1e4:53 :::*                                2665/dnsmasq
udp        0      0 fe80::39e8:1af0:77a7:c1e4:53 :::*                                2665/dnsmasq

UDP连接不多，一直在十几到二十几徘徊。

在passwall2里面关闭UDP代理，使用是一点问题没有，很完美。

@Fangliding
Copy link
Member

打log看了一眼 不管是原来是超时逻辑还是后面加的 都会正常返回 如果真像怀疑的那样就是 leak goroutine 早炸的满天星了
默认的udp回收时间就是5分钟 按你描述就是在5分钟内爆炸的 那就都可以解释的通了 你自己试一下改policy把connIdle改成30之类的比较低的数值看看

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024 •
edited
Loading

@Fangliding 依旧，也将配置手动给成：

...
  "policy": {
    "levels": {
      "0": {
        "statsUserUplink": false,
        "statsUserDownlink": false,
        "connIdle": 10
      }
    }
  },
...
未命名1735235876

哈哈，先睡吧，来日方长

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024 •
edited
Loading

@Fangliding 刚刚评论完，程序发生了异常

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7fa494]

goroutine 28338 [running]:
github.com/xtls/xray-core/proxy/dokodemo.(*DokodemoDoor).Process(0x4000368c40, {0x108f058, 0x4001aee360}, 0x3, {0x1095fd0, 0x4002a56600}, {0x1091520, 0x4004c83380})
        github.com/xtls/xray-core/proxy/dokodemo/dokodemo.go:179 +0x884
github.com/xtls/xray-core/app/proxyman/inbound.(*udpWorker).callback.func1()
        github.com/xtls/xray-core/app/proxyman/inbound/worker.go:331 +0x358
created by github.com/xtls/xray-core/app/proxyman/inbound.(*udpWorker).callback in goroutine 15
        github.com/xtls/xray-core/app/proxyman/inbound/worker.go:308 +0x294

后增：

试了两次，发现都是启动后5-6分钟的时间节点奔溃，好像是什么资源在5分钟后自动回收的代码内存异常，idle时间确实改成了10秒。

@Fangliding
Copy link
Member

https://github.com/XTLS/Xray-core/actions/runs/12507019416

这个呢 还是不行不猜谜了 得以后有时间再弄个透代自己看了

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024

@Fangliding 测试大概7 8分钟，问题依旧
shell 日志：

[root@Kwrt:02:26 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:26:17 CST 2024
661
[root@Kwrt:02:26 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:26:30 CST 2024
823
[root@Kwrt:02:26 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:26:46 CST 2024
974
[root@Kwrt:02:26 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:27:12 CST 2024
1298
[root@Kwrt:02:27 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:28:07 CST 2024
1999
[root@Kwrt:02:28 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:28:31 CST 2024
2292
[root@Kwrt:02:28 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:29:11 CST 2024
2780
[root@Kwrt:02:29 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:29:37 CST 2024
3065
[root@Kwrt:02:29 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:30:15 CST 2024
3532
[root@Kwrt:02:30 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:30:43 CST 2024
3824
[root@Kwrt:02:30 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:32:01 CST 2024
4023
[root@Kwrt:02:32 AM /usr/bin] # date && netstat -anup | grep xray | wc -l
Fri Dec 27 02:32:18 CST 2024
4024

@Fangliding
Copy link
Member

几百个的时候在那晾着会掉下去吗

@jpsn123
Copy link
Author

jpsn123 commented Dec 26, 2024 •
edited
Loading

不会掉，不过我在防火墙那边发现，导致UDP快速增加的原因是BT流量（应该是员工下载东西），但是即使把BT流量禁用掉，UDP也是不停增加，只是速度慢点，大概2秒一个这样子。

后增：

BT禁用一段时间后，UDP也没有释放，且依然在增加。

@Fangliding Fangliding reopened this Dec 27, 2024
@Fangliding
Copy link
Member

我自己测了一下 数千并发核心都可以正常回收两个socket不会有任何泄露问题 复现不了

@RPRX
Copy link
Member

RPRX commented Dec 27, 2024

@Fangliding 所以 main 的代码没问题吗

@Fangliding
Copy link
Member

Fangliding commented Dec 27, 2024 •
edited
Loading

我是在main分支最新的代码上测的 之前我怀疑close有问题没法正确回收但是测了可以 至于先前的修复 我打了俩defer输出日志 两个地方的defer(旧的和新加的)都是可以正确执行 应该不是那的问题 不过我也没测

@RPRX
Copy link
Member

RPRX commented Dec 27, 2024 •
edited
Loading

@Fangliding main 分支那个修复应该还是有些用的，因为 Process() return 时可能下行还在 copy，导致提前清理了、后面的没清理

@RPRX
Copy link
Member

RPRX commented Dec 27, 2024

其实这个以后可以改成烧火棍，就是直接构造 IP 包：#237 (comment)

@jpsn123
Copy link
Author

jpsn123 commented Dec 27, 2024

ket不会有任何泄露问题 复现不了

配置文件有吗，是通过openwrt吗，我研究一下

@jpsn123
Copy link
Author

jpsn123 commented Dec 27, 2024

@Fangliding @RPRX 好消息！！ 当我在把网络中所有BT流量禁用后，并且重新启动xray（idle是10秒），运行10分钟后UDP连接增张到150+，观察十几分钟发现一直在50-200之间振荡。然后我把所有员工上网流量路由到其他网关去（等于断网），此时这个网关没有任何网络活动，一段时候后UDP降到3个。来回测试了很多遍，发现只要断网30s，UDP立刻降到3个，其实这三个都是xray自己的服务端口，也就是说此时不再有任何UDP连接泄露！

@jpsn123
Copy link
Author

jpsn123 commented Dec 27, 2024

@Fangliding @RPRX 晚上再次进行了详细测试，终于被我发现问题的根本原因，就是UDP释放不及时。

正常网络环境中很难察觉异常，但其实如果仔细一点就会发现，绝大部分UDP连接是在断网一段时间后，同时释放，这个其实不应该，因为UDP通常是逐步建立的，所以断网后应该逐步释放。

为了进一步确定问题，我特地配置了一个BT环境，使用的软件是qBittorrent，找了一些资源下载试试，果然UDP连接大量且快速的建立，在xray的日志中可以看到这一点， 我截取日志一段。

2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:180.246.107.145:59687
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:180.246.107.145:59687
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:180.246.107.145:59687
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:180.246.107.145:59687
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:72.21.17.75:52083
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:58.176.179.77:18686
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:113.71.212.130:1031
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:183.109.253.170:28761
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:1.65.165.66:13718
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:175.156.125.57:61495
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:126.217.5.114:25095
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:183.109.253.170:28761
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:154.201.95.21:42761
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:219.70.200.51:22652
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:59.24.156.173:28384
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:121.186.6.49:26982
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:59.149.41.101:31008
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:114.30.11.16:39891
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:45.38.189.6:35585
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:119.237.68.80:45535
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:119.237.68.80:45535
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:119.237.68.80:45535
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:119.237.68.80:45535
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:119.237.68.80:45535
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:119.237.68.80:45535
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:119.237.68.80:45535
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:31.200.249.233:31996
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:110.133.84.190:9300
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:54.77.218.23:6992
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:46.242.11.63:6430
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:101.86.67.150:13352
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:118.150.178.172:20439
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:152.172.129.148:51551
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:115.221.233.115:51413
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:112.20.199.110:22648
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:88.135.252.26:18970
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:58.18.68.70:21000
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:180.246.107.145:59687
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:118.150.178.172:1
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:183.77.40.108:1
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:218.234.7.67:32688
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:101.86.67.150:13352
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:223.16.175.207:32141
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:138.59.128.6:54849
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:188.189.6.209:24175
2024/12/27 22:39:31 [Debug] transport/internet/udp: UDP original destination: udp:38.123.192.6:63638

就这样一直持续十几分钟，很快UDP连接达到4000+，并且开始报错too many open files。
虽然我配置了connIdle=10，但是UDP并没有被释放，即使是第一个建立的UDP，十几分了也没有被释放。报错以后我开始断网，网关中没有任何连接建立（除了ssh），断网10s后没有UDP连接被释放，1分钟以后也没有，直到十分钟以后，UDP连接从4000+，突然掉到3个，也就是说所有UDP都被释放。

就好像新建立UDP给老的UDP续上命了。猜想：defer没有在正确的时间执行，可能是资源被意外引用了，延续了连接的生命周期。

一言以蔽之，只要你能复现出UDP大部分连接是同时释放，那就说明这个bug是存在的。

希望我的测试能够提供给开发者有意义的帮助。

@RPRX
Copy link
Member

RPRX commented Dec 27, 2024 •
edited
Loading

@jpsn123 我觉得你的测试的确是有帮助的，回想我说的

可能 Process() 都 return 了但 responseDone() 还在执行

这个如果极限一点，上行都五分钟超时了，下行还在一直接收数据，就会导致那些 UDP 迟迟没有释放，断网才能释放

但问题是 UDP 又不像 TCP 能知道客户端收没收，可能客户端早就不要了，下行还在一直发给客户端

所以对于 UDP，应当改成仅以上行超时为标准，上行超时了就把下行也打断

@RPRX RPRX reopened this Dec 27, 2024
@RPRX RPRX closed this as completed in ec46a57 Dec 28, 2024
@RPRX RPRX reopened this Dec 28, 2024
@RPRX RPRX closed this as completed in a3dc81e Dec 28, 2024
@RPRX RPRX reopened this Dec 28, 2024
@RPRX RPRX closed this as completed in 8377975 Dec 28, 2024
@RPRX
Copy link
Member

RPRX commented Dec 28, 2024 •
edited
Loading

a8559a1 改成了 UDP 上行超时后打断下行并清理 fake UDP，先看看效果，其实 Socks5 由于 *ray 实现不标准，也应该这么改

关于 task、pipe 等逻辑有点杂乱，以后再理，总之先加了些 runtime.Gosched()，希望能把已经收到本地的 UDP 包传回去

@jpsn123 试试 https://github.com/XTLS/Xray-core/actions/runs/12523589260

@RPRX RPRX reopened this Dec 28, 2024
@RPRX RPRX closed this as completed in a8559a1 Dec 28, 2024
@RPRX
Copy link
Member

RPRX commented Dec 28, 2024

为了加两处注释又开关一次，好壮观的开合跳，看来还是今天太闲，看看能不能把 XUDP 的文章写完然后发一版

@jpsn123
Copy link
Author

jpsn123 commented Dec 28, 2024

@RPRX 试了一下，有一个改进，就是断网后以前是10分钟左右才能收回，现在收回时间就是idle时间。

但是BT环境下，宏观上来看，UDP仍然是直到禁用BT流量后，过idle时间后被突然统一收回。我监控了一个UDP连接ip183.54.205.187 （下方日志首行），整个测试过程中日志只出现一次（是不是可以说明该UDP连接没有流量记录了），但是过了idle时间也没有被释放。

有的UDP连接在整个过程中日志里面记录很多次，有UDP只被记录一次，但是他们在断网前都没有被释放。我不确定他们是不是已经被释放了，后面又被重新打开，还是就是一直没有释放。我这里先贴上测试过程的日志。
xray.log

@RPRX
Copy link
Member

RPRX commented Dec 28, 2024

有的UDP连接在整个过程中日志里面记录很多次，有UDP只被记录一次，但是他们在断网前都没有被释放。

这是预期行为，因为 Xray 会按下行 UDP 包的源地址缓存 fake UDP，就不用反复创建，其它软件比如 Clash 是只用一次就销毁

上行五分钟不活跃后，所有与这次代理相关的 fake UDP 会被一起释放，v25 改成直接构造 IP 包吧

你可以先提升系统 limit，但话说回来，其他人没遇到这个问题可能是他们早就把 BT 禁了，因为可能会封 VPS

@jpsn123
Copy link
Author

jpsn123 commented Dec 28, 2024

好，感谢 @RPRX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@LearZhou @jpsn123 @xqzr @Fangliding @RPRX