国产+高潮+在线,国产 av 仑乱内谢,www国产亚洲精品久久,51国产偷自视频区视频,成人午夜精品网站在线观看

workerman/gateway-worker進(jìn)程跟蹤 restart_syscall(<... resuming interrupted restart_syscall ...>

Tinywan

運(yùn)行狀態(tài)

----------------------------------------------GLOBAL STATUS----------------------------------------------------
Workerman version:3.5.31          PHP version:7.4.16
start time:2021-08-17 14:07:15   run 336 days 21 hours   
load average: 0.02, 0, 0         event-loop:\Workerman\Events\Event
3 workers       7 processes
worker_name    exit_status      exit_count
Register       0                0
BusinessWorker 0                16
BusinessWorker 64000            5
Gateway        0                0
----------------------------------------------PROCESS STATUS---------------------------------------------------
pid memory  listening                  worker_name    connections send_fail timers  total_request qps    status
1957    6M      text://172.24.171.109:1236 Register       6           0         0       7762771       0      [idle]
1962    17.77M  websocket://0.0.0.0:9502   Gateway        30          913913    5       117133951     0      [idle]
1963    17.77M  websocket://0.0.0.0:9502   Gateway        21          920443    5       117666307     0      [idle]
8701    N/A     none         BusinessWorker N/A         N/A       N/A     N/A           N/A    [busy] 
9574    10M     none                       BusinessWorker 3           0         1       23086407      0      [idle]
9576    10M     none                       BusinessWorker 3           0         1       23071418      0      [idle]
10215   10M     none                       BusinessWorker 3           0         1       1024          0      [idle]
----------------------------------------------PROCESS STATUS---------------------------------------------------
Summary 70M     -                          -              66          1834356   13      288721878     0      [Summary]

8701 進(jìn)程跟蹤

8701 N/A none BusinessWorker N/A N/A N/A N/A N/A [busy]

strace -ttp 8701
strace: Process 8701 attached
11:30:30.102970 restart_syscall(<... resuming interrupted restart_syscall ...>

) = 0
11:30:51.310705 poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
11:30:51.310797 poll([{fd=12, events=POLLIN|POLLERR|POLLHUP}], 1, 60000^Cstrace: Process 8701 detached
 <detached ...>

進(jìn)程跟蹤一直被阻塞上面這種情況，無(wú)任何響應(yīng)

按道理應(yīng)該是下面這種才是正常的

11:41:44.166569 recvfrom(12, "+OK\r\n", 8192, MSG_DONTWAIT, NULL, NULL) = 5
11:41:44.166655 poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
11:41:44.166744 sendto(12, "*3\r\n$4\r\nHGET\r\n$10\r\nlive_layer\r\n$"..., 45, MSG_DONTWAIT, NULL, 0) = 45
11:41:44.166830 poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
11:41:44.166908 poll([{fd=12, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=12, revents=POLLIN}])
11:41:44.168704 recvfrom(12, "$257\r\n{\"msg_type\":\"layer_message"..., 8192, MSG_DONTWAIT, NULL, NULL) = 265
11:41:44.168806 poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
11:41:44.168887 sendto(12, "*3\r\n$4\r\nHGET\r\n$10\r\nlive_layer\r\n$"..., 45, MSG_DONTWAIT, NULL, 0) = 45
11:41:44.168989 poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
11:41:44.169069 poll([{fd=12, events=POLLIN|POLLERR|POLLHUP}], 1, 60000) = 1 ([{fd=12, revents=POLLIN}])
11:41:44.170843 recvfrom(12, "$254\r\n{\"msg_type\":\"layer_message"..., 8192, MSG_DONTWAIT, NULL, NULL) = 262
11:41:44.170977 sendto(11, "\0\0\1\271\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0j\1\0\0\0\0\0\0{\"ms"..., 441, 0, NULL, 0) = 441
11:41:44.171176 alarm(0)                = 30
11:41:44.172994 recvfrom(10, "\0\0\0l\3\254\30\253m\7\320\254\30\253m\341N\0\0\0\27\1%\36\0\0\0=a:2:"..., 65535, 0, NULL, NULL) = 108
11:41:44.173124 alarm(30)               = 0
11:41:44.173218 close(12)               = 0
11:41:44.173300 stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=362, ...}) = 0
11:41:44.173387 openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 12
11:41:44.173467 fstat(12, {st_mode=S_IFREG|0644, st_size=277, ...}) = 0
11:41:44.173544 read(12, "127.0.0.1\tlocalhost\n\n# The follo"..., 4096) = 277
11:41:44.173624 read(12, "", 4096)      = 0
11:41:44.173699 close(12)               = 0
11:41:44.173807 socket(AF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 12
11:41:44.173889 connect(12, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.53")}, 16) = 0
11:41:44.173991 poll([{fd=12, events=POLLOUT}], 1, 0) = 1 ([{fd=12, revents=POLLOUT}])
11:41:44.174072 sendto(12, ":\271\1\0\0\1\0\0\0\0\0\0\24r-bp1ob5zk2hcwfs58n"..., 61, MSG_NOSIGNAL, NULL, 0) = 61
11:41:44.174233 poll([{fd=12, events=POLLIN}], 1, 2000) = 1 ([{fd=12, revents=POLLIN}])
11:41:44.174327 ioctl(12, FIONREAD, [77]) = 0
1959 4 1
4個(gè)回答
walkor 打賞

運(yùn)行 lsof -np 8701 看下fd為12的資源是什么

  • Tinywan 2022-07-20

    業(yè)務(wù)緊急，我直接給重啟了
Tinywan

workerman.log 查看日志

2022-07-20 11:29:39 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:39 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:40 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:40 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:41 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:41 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:41 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:41 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:41 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:42 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:42 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:42 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:42 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:42 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:43 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:44 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:44 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:44 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:45 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:45 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:45 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:46 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:47 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:47 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:47 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:48 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:48 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:48 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:49 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:49 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:50 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:50 pid:1962 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
2022-07-20 11:29:51 pid:1963 SendBufferToWorker fail. May be the send buffer are overflow. See http://doc2.workerman.net/send-buffer-overflow.html
  • walkor 2022-07-20

    這個(gè)應(yīng)該是businessworker進(jìn)程卡在poll([{fd=12,那里導(dǎo)致的，那個(gè)businessworker進(jìn)程長(zhǎng)時(shí)間無(wú)法接收gateway發(fā)過(guò)來(lái)的信息
  • Tinywan 2022-07-20

    嗯，我下次按照這個(gè)手冊(cè)排查一下，http://www.wtbis.cn/doc/gateway-worker/send-buffer-overflow.html
  • Tinywan 2022-11-18

    這個(gè)問(wèn)題又出現(xiàn)了了

    lsof -np 10261
COMMAND   PID USER   FD      TYPE    DEVICE SIZE/OFF      NODE NAME
php     10261  www  rtd       DIR     252,1     4096         2 /
php     10261  www  txt       REG     252,1 53003800   1452431 /usr/local/php-7.4/bin/php
php     10261  www  mem       REG     252,1    97176    401620 /lib/x86_64-linux-gnu/libnsl-2.27.so
php     10261  www  mem       REG     252,1    47576    401625 /lib/x86_64-linux-gnu/libnss_nis-2.27.so
php     10261  www  mem       REG     252,1    39744    401621 /lib/x86_64-linux-gnu/libnss_compat-2.27.so
php     10261  www  mem       REG     252,1    26936    401622 /lib/x86_64-linux-gnu/libnss_dns-2.27.so
php     10261  www  mem       REG     252,1    47568    401623 /lib/x86_64-linux-gnu/libnss_files-2.27.so
php     10261  www  mem       REG     252,1   913448   1580661 /usr/local/libevent-2.1.12/lib/libevent_core-2.1.so.7.0.1
php     10261  www  mem       REG     252,1   527336   1580665 /usr/local/libevent-2.1.12/lib/libevent_extra-2.1.so.7.0.1
php     10261  www  mem       REG     252,1    98952   1580673 /usr/local/libevent-2.1.12/lib/libevent_openssl-2.1.so.7.0.1
php     10261  www  mem       REG     252,1  1033840   1322683 /usr/local/php-7.4/lib/php/extensions/no-debug-non-zts-20190902/event.so
php     10261  www  mem       REG     252,1  2850040   1322630 /usr/local/php-7.4/lib/php/extensions/no-debug-non-zts-20190902/redis.so
php     10261  www  mem       REG     252,1  3004224   1053795 /usr/lib/locale/locale-archive
php     10261  www  mem       REG     252,1    39208    401615 /lib/x86_64-linux-gnu/libcrypt-2.27.so
php     10261  www  mem       REG     252,1   300888   1062361 /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
php     10261  www  mem       REG     252,1    60400   1062355 /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
php     10261  www  mem       REG     252,1   165880   1062359 /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
php     10261  www  mem       REG     252,1    31032   1050749 /usr/lib/x86_64-linux-gnu/libffi.so.6.0.4
php     10261  www  mem       REG     252,1    88680   1062348 /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
php     10261  www  mem       REG     252,1   217560   1062357 /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
php     10261  www  mem       REG     252,1   661696   1062353 /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
php     10261  www  mem       REG     252,1   573464   1062363 /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
php     10261  www  mem       REG     252,1    35360   1062365 /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
php     10261  www  mem       REG     252,1    14256    401498 /lib/x86_64-linux-gnu/libkeyutils.so.1.5
php     10261  www  mem       REG     252,1    75776   1050819 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.5.5
php     10261  www  mem       REG     252,1  1237640   1049002 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0
php     10261  www  mem       REG     252,1    84032    393675 /lib/x86_64-linux-gnu/libgpg-error.so.0.22.0
php     10261  www  mem       REG     252,1   265712   1062367 /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
php     10261  www  mem       REG     252,1   109296   1062369 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
php     10261  www  mem       REG     252,1    43616   1051640 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
php     10261  www  mem       REG     252,1    14248    393352 /lib/x86_64-linux-gnu/libcom_err.so.2.1
php     10261  www  mem       REG     252,1   199104   1051642 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
php     10261  www  mem       REG     252,1   877056   1051638 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
php     10261  www  mem       REG     252,1   526688   1050765 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.2
php     10261  www  mem       REG     252,1   219304   1050801 /usr/lib/x86_64-linux-gnu/libnettle.so.6.4
php     10261  www  mem       REG     252,1   211704   1050773 /usr/lib/x86_64-linux-gnu/libhogweed.so.4.4
php     10261  www  mem       REG     252,1  1461856   1049007 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.14.10
php     10261  www  mem       REG     252,1  1562664   1050823 /usr/lib/x86_64-linux-gnu/libunistring.so.2.1.0
php     10261  www  mem       REG     252,1  1159864    393318 /lib/x86_64-linux-gnu/libgcrypt.so.20.2.1
php     10261  www  mem       REG     252,1 26904112   1050728 /usr/lib/x86_64-linux-gnu/libicudata.so.60.2
php     10261  www  mem       REG     252,1    55544   1050152 /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.10.8
php     10261  www  mem       REG     252,1   327024   1050153 /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.10.8
php     10261  www  mem       REG     252,1   305456   1051635 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
php     10261  www  mem       REG     252,1    55136   1061308 /usr/lib/x86_64-linux-gnu/libpsl.so.5.2.0
php     10261  www  mem       REG     252,1   113584   1062378 /usr/lib/x86_64-linux-gnu/librtmp.so.1
php     10261  www  mem       REG     252,1   116656   1048672 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.3
php     10261  www  mem       REG     252,1   153352   1062376 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.15.2
php     10261  www  mem       REG     252,1   153984    393687 /lib/x86_64-linux-gnu/liblzma.so.5.2.2
php     10261  www  mem       REG     252,1   144976    401628 /lib/x86_64-linux-gnu/libpthread-2.27.so
php     10261  www  mem       REG     252,1  2030928    401613 /lib/x86_64-linux-gnu/libc-2.27.so
php     10261  www  mem       REG     252,1    96616    401608 /lib/x86_64-linux-gnu/libgcc_s.so.1
php     10261  www  mem       REG     252,1    80328   1063298 /usr/lib/x86_64-linux-gnu/libzip.so.4.0.0
php     10261  www  mem       REG     252,1    87912   1063117 /usr/lib/x86_64-linux-gnu/libexslt.so.0.8.17
php     10261  www  mem       REG     252,1   247952   1063118 /usr/lib/x86_64-linux-gnu/libxslt.so.1.1.29
php     10261  www  mem       REG     252,1  1082648   1050774 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
php     10261  www  mem       REG     252,1   496784   1063281 /usr/lib/x86_64-linux-gnu/libonig.so.4.0.0
php     10261  www  mem       REG     252,1  1796104   1050770 /usr/lib/x86_64-linux-gnu/libicuuc.so.60.2
php     10261  www  mem       REG     252,1  2754872   1050756 /usr/lib/x86_64-linux-gnu/libicui18n.so.60.2
php     10261  www  mem       REG     252,1    55304   1050760 /usr/lib/x86_64-linux-gnu/libicuio.so.60.2
php     10261  www  mem       REG     252,1   735704   1050817 /usr/lib/x86_64-linux-gnu/libfreetype.so.6.15.0
php     10261  www  mem       REG     252,1   424648   1063485 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.1.2
php     10261  www  mem       REG     252,1   202672   1061305 /usr/lib/x86_64-linux-gnu/libpng16.so.16.34.0
php     10261  www  mem       REG     252,1   518600   1062379 /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0
php     10261  www  mem       REG     252,1   116960    393760 /lib/x86_64-linux-gnu/libz.so.1.2.11
php     10261  www  mem       REG     252,1  2917216   1062948 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1
php     10261  www  mem       REG     252,1   577312   1062957 /usr/lib/x86_64-linux-gnu/libssl.so.1.1
php     10261  www  mem       REG     252,1  1834232   1050779 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.4
php     10261  www  mem       REG     252,1    14560    401616 /lib/x86_64-linux-gnu/libdl-2.27.so
php     10261  www  mem       REG     252,1  1700792    401617 /lib/x86_64-linux-gnu/libm-2.27.so
php     10261  www  mem       REG     252,1    66728    393641 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
php     10261  www  mem       REG     252,1  1594864   1050839 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
php     10261  www  mem       REG     252,1    31680    401630 /lib/x86_64-linux-gnu/librt-2.27.so
php     10261  www  mem       REG     252,1    97072    401629 /lib/x86_64-linux-gnu/libresolv-2.27.so
php     10261  www  mem       REG     252,1   179152    401609 /lib/x86_64-linux-gnu/ld-2.27.so
php     10261  www  mem       REG     252,1    26376   1051162 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
php     10261  www    0u      CHR     136,2      0t0         5 /dev/pts/2 (deleted)
php     10261  www    1w      CHR       1,3      0t0         6 /dev/null
php     10261  www    2w      CHR       1,3      0t0         6 /dev/null
php     10261  www    3u     IPv4 668010317      0t0       TCP 172.24.171.109:37484->172.30.237.31:6379 (ESTABLISHED)
php     10261  www    4u      CHR     136,2      0t0         5 /dev/pts/2 (deleted)
php     10261  www    5u  a_inode      0,13        0     10598 [eventpoll]
php     10261  www    6r     FIFO      0,12      0t0 668010669 pipe
php     10261  www    7w     FIFO      0,12      0t0 668010669 pipe
php     10261  www    8u     IPv4 668010670      0t0       TCP 172.24.171.109:52992->172.24.171.109:rmtcfg (ESTABLISHED)
php     10261  www    9u     IPv4 668010678      0t0       TCP 172.24.171.109:37496->172.30.237.31:6379 (ESTABLISHED)
php     10261  www   10u     IPv4 668010685      0t0       TCP 172.24.171.109:57894->172.24.171.109:cisco-sccp (ESTABLISHED)
php     10261  www   11u     IPv4 668010686      0t0       TCP 172.24.171.109:39910->172.24.171.109:2001 (ESTABLISHED)
php     10261  www   12u     IPv4 999893004      0t0       TCP 172.24.171.109:52572->172.30.237.31:6379 (ESTABLISHED)
  • Tinywan 2022-11-18

    內(nèi)存和CPU使用情況

    Tasks: 105 total,   1 running,  68 sleeping,   0 stopped,   0 zombie
%Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  8021284 total,  3142400 free,   358348 used,  4520536 buff/cache
KiB Swap:        0 total,        0 free,        0 used.  7333556 avail Mem
  • Tinywan 2022-11-18

    進(jìn)程跟蹤

    # sudo strace -ttp 10261
strace: Process 10261 attached
13:52:26.597217 restart_syscall(<... resuming interrupted poll ...>

) = 0
13:53:13.815227 poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout)
13:53:13.815302 poll([{fd=12, events=POLLIN|POLLERR|POLLHUP}], 1, 60000
  • Tinywan 2022-11-18

    用lsof -p $pid | grep $fd 查看進(jìn)程在等待哪個(gè)外部資源的返回

    lsof -p 10261 | grep 12
php     10261  www  mem       REG     252,1   913448   1580661 /usr/local/libevent-2.1.12/lib/libevent_core-2.1.so.7.0.1
php     10261  www  mem       REG     252,1   527336   1580665 /usr/local/libevent-2.1.12/lib/libevent_extra-2.1.so.7.0.1
php     10261  www  mem       REG     252,1    98952   1580673 /usr/local/libevent-2.1.12/lib/libevent_openssl-2.1.so.7.0.1
php     10261  www  mem       REG     252,1  1237640   1049002 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0
php     10261  www  mem       REG     252,1   265712   1062367 /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
php     10261  www  mem       REG     252,1 26904112   1050728 /usr/lib/x86_64-linux-gnu/libicudata.so.60.2
php     10261  www  mem       REG     252,1    87912   1063117 /usr/lib/x86_64-linux-gnu/libexslt.so.0.8.17
php     10261  www  mem       REG     252,1   577312   1062957 /usr/lib/x86_64-linux-gnu/libssl.so.1.1
php     10261  www    6r     FIFO      0,12      0t0 668010669 pipe
php     10261  www    7w     FIFO      0,12      0t0 668010669 pipe
php     10261  www   12u     IPv4 999893004      0t0       TCP iZbp1hoizi4geybteffl6zZ:52572->172.30.237.31:6379 (ESTABLISHED)
  • walkor 2022-11-18

    看起來(lái)是在等172.30.237.31:6379這個(gè)地址返回?cái)?shù)據(jù)。redis用得是自建redis還是阿里云
  • Tinywan 2022-11-18

    阿里云的
  • walkor 2022-11-18

    阿里云的redis最好用個(gè)定時(shí)器，定時(shí)發(fā)一個(gè)心跳數(shù)據(jù)，維持連接，時(shí)間下小于60秒，比如55。
    阿里云redis可能會(huì)在redis連接空閑1分鐘后清理連接，不發(fā)fin包通知那種，導(dǎo)致redis擴(kuò)展無(wú)法知道連接已經(jīng)不可用，認(rèn)為連接仍然存活，但是數(shù)據(jù)一直收不到。
  • Tinywan 2022-11-18

    嗯。目前排查阿里云Redis那邊確實(shí)是過(guò)一會(huì)就會(huì)釋放連接，但是gateway這邊還一直在嘗試連接。感謝！群主
  • yohe 2022-11-18

    1
  • changepll 2022-11-18

    請(qǐng)問(wèn)阿里云redis加定時(shí)器, 是指在定時(shí)任務(wù)中加一個(gè)每55秒連一下隨意查個(gè)key意思嗎
  • walkor 2022-11-18

    對(duì)
Tinywan

找到相關(guān)的帖子了：在阿里云上遇到一個(gè)奇怪的 Redis 連接問(wèn)題，每隔十來(lái)分鐘，服務(wù)里的 Redis client 庫(kù)就報(bào)告連接 Redis server 超時(shí)，當(dāng)時(shí)花了很大功夫，發(fā)現(xiàn)是阿里云會(huì)斷開(kāi)長(zhǎng)時(shí)間閑置的 TCP 連接，不給兩頭發(fā) FIN or RST 包

差不多一兩年前，在阿里云上遇到一個(gè)奇怪的 Redis 連接問(wèn)題，每隔十來(lái)分鐘，服務(wù)里的 Redis client 庫(kù)就報(bào)告連接 Redis server 超時(shí)，當(dāng)時(shí)花了很大功夫，發(fā)現(xiàn)是阿里云會(huì)斷開(kāi)長(zhǎng)時(shí)間閑置的 TCP 連接，不給兩頭發(fā) FIN or RST 包，而當(dāng)時(shí)我們的 Redis server 沒(méi)有打開(kāi) tcp_keepalive 選項(xiàng)，于是 Redis server 側(cè)那個(gè)連接還存在于 Linux conntrack table 里，而 Redis client 側(cè)由于連接池重用連接進(jìn)行 get、set 發(fā)現(xiàn)連接壞掉就關(guān)閉了，所以 client 側(cè)的對(duì)應(yīng) local port 回收了，當(dāng)接下來(lái) Redis 重用這個(gè) local port 向 Redis server 發(fā)起連接時(shí)，由于 Redis server 側(cè)的 conntrack table 里 <client_ip, client_port, redis-server, 6379> 四元組對(duì)應(yīng)狀態(tài)是 ESTABLISHED，所以自然客戶端發(fā)來(lái)的 TCP SYN packet 被丟棄，Redis client 看到的現(xiàn)象就是連接超時(shí)。

https://zhuanlan.zhihu.com/p/52622856

  • 暫無(wú)評(píng)論
admin

真好，利用大佬的教訓(xùn)，免費(fèi)給我們培訓(xùn)

年代過(guò)于久遠(yuǎn)，無(wú)法發(fā)表回答
??