Investigate TCP Keepalives

[ustulation]

Ideally we would want to not have app level heartbeat if protocol level keepalives are sufficient. However the API for it is not very flexible at this time in mio and allows to set the duration after which the 1st keepalive should be sent when the socket is idling, but not how much interval needs to elapse before we decide to sever the connection or how many keepalives we send after the first keepalive. Depending on the duration of keepalive-interval (the 1st keepalive that is sent) other parameters seem to be fixed to reasonable defaults as noted here.

1. Investigate this and also see if we have an API either from mio or std::net that allows a bit more flexibility here
2. Find out if the behaviour is similar on other platforms (Windows, OS X, iOS and Android) and report here

[povilasb]

1. neither mio nor std has the API flexible enough to control TCP socket keepalives, but this can be done with libc crate. I've done some prototyping:

/// Set TCP keep alive options for a given socket, if configured.
pub fn set_keep_alive(stream: &TcpStream, conf: &SocketConfig) -> io::Result<()> {
    if let Some((idle, interval, count)) = conf.keep_alive {
        stream.set_keepalive(Some(Duration::from_secs(u64::from(idle))))?;
        let fd = stream.as_raw_fd();
        set_ip_opt(fd, libc::TCP_KEEPINTVL, interval)?;
        set_ip_opt(fd, libc::TCP_KEEPCNT, count)?;
    }
    Ok(())
}

/// Sets IP level option for a given socketlevel option for a given socket
fn set_ip_opt(sock_fd: RawFd, opt: libc::c_int, val: u32) -> io::Result<()> {
    unsafe {
        let optval: libc::c_int = val as libc::c_int;
        let ret = libc::setsockopt(
            sock_fd,
            libc::IPPROTO_TCP,
            opt,
            &optval as *const _ as *const libc::c_void,
            mem::size_of_val(&optval) as libc::socklen_t,
        );
        if ret != 0 {
            Err(io::Error::last_os_error())
        } else {
            Ok(())
        }
    }
}

That could also be done with the nix crate which provides a higher level API once nix-rust/nix#889 gets merged.

[povilasb]

Windows does not support the TCP_KEEPINTVL and TCP_KEEPCNT options. But there's an alternative API:

1. set SO_KEEPALIVE for socket: https://docs.microsoft.com/en-us/windows/desktop/WinSock/so-keepalive
2. then configure keep alive intervals with: https://msdn.microsoft.com/en-us/library/windows/desktop/dd877220%28v=vs.85%29.aspx

See example: http://read.pudn.com/downloads79/ebook/301417/Chapter09/SIO_KEEPALIVE_VALS/alive.c__.htm

[povilasb]

It's weird, on Mac OS seems like TCP_KEEPINTVL and CNT are documented: https://github.com/apple/darwin-xnu/blob/xnu-4570.1.46/bsd/man/man4/tcp.4#L190, but when trying to use from code, it says the symbols are not found. Or maybe that's just libc wrappers... Should also see what the constants are and try to use them directly.

[povilasb]

We won't be using TCP keepalive functionality any time soon. Now, it's obvious that it's configuration differs on each platform and it only works with TCP, but socket-collection also supports UDP based protocols. So are postponing this research to the future.