• Audit POSIX close-on-exec (FD_CLOEXEC/O_CLOEXEC): fork/exec children i

    From Rob Swindell@1:103/705 to GitLab issue in main/sbbs on Wed Jun 24 01:12:56 2026
    open https://gitlab.synchro.net/main/sbbs/-/issues/1174

    ## Summary

    Follow-up to #1151 (the Windows listen-socket inheritance leak, fixed in commit 6816ce611). That fix addressed Windows handle inheritance; the equivalent **POSIX `fork`/`exec` close-on-exec story should be audited and hardened separately**, as flagged in the original report.

    On POSIX, a child created by `fork()` inherits **every** open file descriptor of the parent. Unless those fds are marked close-on-exec (`FD_CLOEXEC` / `O_CLOEXEC` / `SOCK_CLOEXEC`) or explicitly closed in the child before `exec`, the exec'd program inherits them — including all server listen sockets, the active client socket, message-base fds, SpiderMonkey/cryptlib fds, etc. This is the direct analog of the Windows leak: a long-lived external (a timed-event door, a CGI) could keep server listen sockets bound after the parent exits.

    ## Current state

    - **Zero** uses of `O_CLOEXEC`, `FD_CLOEXEC`, or `SOCK_CLOEXEC` across the entire `src/` tree (`git grep` returns nothing).
    - `sbbs3/xtrn.cpp` `external()` POSIX path: after `fork()` (xtrn.cpp:1021) the child sets up stdio redirection via `dup2()` and redirects unused stdio to `/dev/null`, then `execvp()`s (xtrn.cpp:1862) **without closing any other inherited descriptors** — no `closefrom()`, no fd-range close loop. So the door inherits all of the parent's other fds.
    - `sbbs3/js_global.cpp:52` `execv("/proc/self/exe", ...)` (self-restart) has the same exposure.
    - Listen sockets are created in the shared `xpdev/multisock.c` path (`socket()` at `xpms_add`) with no `SOCK_CLOEXEC`.

    ## Suggested approach

    Mirror the Windows fix's "close at the source" strategy, which is more robust than per-exec cleanup:

    1. **Create long-lived fds close-on-exec.** Set `SOCK_CLOEXEC` when creating listen/accept sockets in `multisock.c` (and `accept4(..., SOCK_CLOEXEC)` where available), and `O_CLOEXEC` on long-lived `open()`s. Where the platform lacks the atomic flag, fall back to `fcntl(fd, F_SETFD, FD_CLOEXEC)` immediately after creation.
    2. **Belt-and-suspenders in the child**, for fds we don't control: after `fork()` and after the intended `dup2()` redirections, `closefrom(3)` (or a portable fd-range close) before `exec`, keeping only 0/1/2 and any fd the door is explicitly meant to receive (e.g. a socket-handle door passed its descriptor).

    Care is needed not to break doors that are *intended* to inherit a specific descriptor (the POSIX equivalent of the Windows passthru/`client_socket_dup` socket-handle door) — those fds must be exempted from the close.

    ## Platform

    POSIX (Linux/macOS/*BSD). The Windows half is resolved by #1151 / 6816ce611.

    — *Authored by Claude (Claude Code), on behalf of @rswindell*
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell@1:103/705 to GitLab note in main/sbbs on Fri Aug 7 15:23:53 2026
    https://gitlab.synchro.net/main/sbbs/-/issues/1174#note_10056

    This is no longer hypothetical. It caused a production outage today (2026-08-07) on the development host, in exactly the shape predicted above:
    "a long-lived external could keep server listen sockets bound after the parent exits."

    ### What happened

    A `systemctl reload` of `sbbs.service` (SIGHUP, recycle-all) was issued at about 14:58. The Terminal Server recycled cleanly, but the Mail, FTP and Services servers could not rebind and looped on their retry counters:

    ```
    mail 0009 !ERROR 98 binding SMTP Transfer Agent socket to port 25: Address already in use
    srvc 0023 !ERROR 98 binding NNTP socket to port 119: Address already in use
    ftp 0014 !ERROR 98 binding FTP Server socket to port 21: Address already in use
    systemd[1]: sbbs.service: Reload operation timed out. Killing reload process. ```

    SMTP and NNTP stayed down for roughly 25 minutes.

    ### What was holding the ports

    A `/bin/bash -i`, running as the `sbbs` user and parented to the daemon, started at 14:26:39, i.e. about half an hour before the recycle. Its environment identifies it as an external program on a node rather than
    anything administrative:

    ```
    SBBSNNUM=11 SBBSNODE=/sbbs/node11/ SBBSCTRL=/sbbs/ctrl/ TERM=pcansi
    ```

    It held 166 open descriptors. Matching the listening-socket inodes from `/proc/net/tcp` against that process's `/proc/<pid>/fd` confirmed it held
    every one of the six affected listen sockets (three ports, two interfaces each):

    | port | inodes held by the door |
    |---|---|
    | 21 | 553856839, 553860416 |
    | 25 | 553856845, 553856847 |
    | 119 | 553861367, 553861369 |

    The rest was mostly dozens of duplicate handles on `ctrl/node.dab`, which is
    a second consequence of the same inheritance and worth keeping in mind for
    the lock-manager behavior.

    Terminating that one process released the ports.

    ### The symptom is worse than "bind failed"

    From a client's point of view this does not look like a down server, and that is what makes it nasty to diagnose. The parent still had its own listen
    sockets open, so the kernel completed TCP handshakes normally: connections
    were accepted into the backlog and simply never processed, because the server threads that would `accept()` them had exited. `ss` showed a Recv-Q of 10 on the FTP listener. Connecting to port 25 gave a successful TCP session and then silence, with no `220` banner at all. It presents as a hung server rather than a bind error, and only the server log names the real cause.

    ### Which of the two suggested remedies would have prevented it

    Only the first one. The door was spawned roughly 30 minutes before the
    recycle, so no cleanup performed at recycle time could have helped; the descriptors were already duplicated into a process the server does not
    control and cannot reach. Creating the listen sockets close-on-exec in `multisock.c` is what closes this, because it acts at the moment the door is exec'd. The `closefrom()` belt-and-suspenders in `external()` would also have covered this particular case, but it only protects children the BBS spawns itself, so the socket-creation fix is the one that generalizes.

    One practical note in favor of doing the socket half first: it is a small, self-contained change, and it carries none of the risk of the child-side
    close, which has to preserve whatever descriptor a socket-handle door is
    meant to inherit.

    Line numbers have drifted since this issue was filed. On master today the `fork()` in `external()` is at `sbbs3/xtrn.cpp:1047` and the `execvp()` at `sbbs3/xtrn.cpp:1905`, and between them the child still touches only fds 0,
    1, 2 and its pipes. `git grep` for `O_CLOEXEC`, `FD_CLOEXEC` and
    `SOCK_CLOEXEC` across `src/` still returns nothing.

    *Authored by Claude (Claude Code), on behalf of @rswindell*
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell@1:103/705 to GitLab note in main/sbbs on Sat Aug 8 18:08:52 2026
    https://gitlab.synchro.net/main/sbbs/-/issues/1174#note_10090

    Closing. Every fork/exec path in the servers now closes what the child has no business with, and the descriptors that survive are stdio plus, for a native socket-door, the one socket it is handed by number.

    ### What landed

    | commit | |
    |---|---|
    | 0db85afb4c (social-29-army, 2026-08-07) | listen sockets close-on-exec on POSIX |
    | b96da4eb92 (forget-6-zone, 2026-08-07) | use the creation flag where the platform has one, closing the window between socket() and fcntl() |
    | f7c74bcbbf (boost-4-denial, 2026-08-08) | accepted client sockets, via accept4() |
    | b16969edb8 (rats-4-fitting, 2026-08-08) | close everything else in the external() child |
    | 7384ba6569 (centers-11-chains, 2026-08-08) | do that with closefrom() rather than close_range() |
    | 0791f3e3bf (spec-12-defeat, 2026-08-08) | the same for the CGI child in websrvr.cpp |

    The Windows half was already done in 6816ce611a (carries-19-baby, 2026-06-24) under #1151.

    ### The one deliberate exemption

    `main.cpp` accepts the node passthru socket with plain `accept()`, not `xp_accept()`, and `external()` exempts it from the close. A native socket-door is handed that descriptor by number through the `%H` command-line specifier, so it is the one descriptor that has to survive exec. Both sites say so, because each otherwise reads as an oversight.

    ### Not done, and why it is no longer a hole

    Marking the long-lived `open()` call sites `O_CLOEXEC` was the fourth item under
    the suggested approach. It was not done and is not needed for correctness. The child-side close covers every descriptor regardless of who opened it, including the ones libraries open where there is no call site to mark. What `O_CLOEXEC` would still add is coverage for an exec occurring inside a library, where there is no child of ours in which to close anything. That is a narrow case and is better raised on its own if it ever bites.

    Worth recording for whoever meets that case: per-site marking cannot be completed by audit. This tree has roughly 1700 `open()`/`fopen()` sites, and the
    process holds descriptors it never opened - four sockets to the MQTT broker, created inside libmosquitto, were the example that settled the question.

    ### Verification

    Each change was checked against a scratch BBS rather than by inspection, using an external program, a socket-door, and a CGI over HTTP that each dumped the descriptor table they were handed. Before: a door started with the node.dab handles, the node log, client sockets and a pipe. After: stdio, and the socket door additionally reported its `%H` descriptor open and usable. For the CGI, a listen socket was deliberately left inheritable to stand in for a library-opened descriptor; the stock child inherited it and the fixed one did not.

    The outage that prompted this is written up in the note above, from 2026-08-07.

    *Authored by Claude (Claude Code), on behalf of @rswindell*
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell@1:103/705 to GitLab issue in main/sbbs on Sat Aug 8 18:08:59 2026
    close https://gitlab.synchro.net/main/sbbs/-/issues/1174
    --- SBBSecho 3.37-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)