Revert "linux: eliminate a read on eventfd per wakeup (#4400)" (#4585)

This reverts commit e5cb1d3d3d4ab3178ac567fb6a7f0f4b5eef3083.

Reason: bisecting says it breaks dnstap.

Revert "linux: eliminate a read on eventfd per wakeup (#4400)" (#4585)

This reverts commit e5cb1d3d3d4ab3178ac567fb6a7f0f4b5eef3083.

Reason: bisecting says it breaks dnstap.

Also revert commit 27134547ff ("kqueue: use EVFILT_USER for async if
available") because otherwise the first commit doesn't revert cleanly,
with enough conflicts in src/unix/async.c that I'm not comfortable
fixing those up manually.

Fixes: https://github.com/libuv/libuv/issues/4584

show more ...

kqueue: disallow ill-suited file descriptor kinds (#4513)

Follows up on https://github.com/libuv/libuv/pull/659.

Signed-off-by: Andy Pan <i@andypan.me>

kqueue: use EVFILT_USER for async if available

Establishes a user event for kqueue to eliminate the overhead
of the pipe and the system call read(2) per wakeup event.

---------

kqueue: use EVFILT_USER for async if available

Establishes a user event for kqueue to eliminate the overhead
of the pipe and the system call read(2) per wakeup event.

---------

Signed-off-by: Andy Pan <i@andypan.me>

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

show more ...

freebsd: fix build on non-intel archs (#4276)

KINFO_FILE_SIZE is only defined on Intel archs.

Fixes: https://github.com/libuv/libuv/issues/4274

freebsd: fix F_KINFO file path handling (#4256)

The new F_KINFO flag does not seem to work with directories nor with
deleted entries.

Fixes: https://github.com/libuv/libuv/issue

freebsd: fix F_KINFO file path handling (#4256)

The new F_KINFO flag does not seem to work with directories nor with
deleted entries.

Fixes: https://github.com/libuv/libuv/issues/4255

show more ...

freebsd: get fs event path with fcntl(F_KINFO)

unix: match kqueue and epoll code (#4091)

Match the implementation for linux.c to kqueue.c in the code around the
calls to kevent and epoll.

In linux.c the code was made more DR

unix: match kqueue and epoll code (#4091)

Match the implementation for linux.c to kqueue.c in the code around the
calls to kevent and epoll.

In linux.c the code was made more DRY by moving the nfds check up
(including a comment of why it's possible) and combining two if checks
into one.

In kqueue.c the assert to check the timeout when nfds == 0 has been
moved to be called directly after the EINTR check. Since it should
always be true regardless.

Ref: https://github.com/libuv/libuv/pull/3893
Ref: https://github.com/nodejs/node/issues/48490

show more ...

unix,win: replace QUEUE with struct uv__queue (#4022)

Recent versions of gcc have started emitting warnings about the liberal
type casting inside the QUEUE macros. Although the warnings

unix,win: replace QUEUE with struct uv__queue (#4022)

Recent versions of gcc have started emitting warnings about the liberal
type casting inside the QUEUE macros. Although the warnings are false
positives, let's use them as the impetus to switch to a type-safer and
arguably cleaner approach.

Fixes: https://github.com/libuv/libuv/issues/4019

show more ...

src: fix events/events_waiting metrics counter (#3957)

The worker pool calls all callbacks locally within the queue. So the
value of nevents doesn't properly reflect that case. Increase

src: fix events/events_waiting metrics counter (#3957)

The worker pool calls all callbacks locally within the queue. So the
value of nevents doesn't properly reflect that case. Increase the number
of events directly from the worker pool's callback to correct this.

In order to properly determine if the events_waiting counter needs to be
incremented, store the timeout value at the time the event provider was
called.

show more ...

src: switch to use C11 atomics where available (#3950)

Switch all code in unix/ to use C11 atomics directly.

Change uv_library_shutdown() to use an exchange instead of load/store.

src: switch to use C11 atomics where available (#3950)

Switch all code in unix/ to use C11 atomics directly.

Change uv_library_shutdown() to use an exchange instead of load/store.

Unfortunately MSVC only started supporting C11 atomics in VS2022 version
17.5 Preview 2 as experimental. So resort to using the Interlocked API.

Ref: https://devblogs.microsoft.com/cppblog/c11-atomics-in-visual-studio-2022-version-17-5-preview-2/
Fixes: https://github.com/libuv/libuv/issues/3948

show more ...

darwin,process: feed kevent the signal to reap children (#3893)

Since we are emulating this event, but are not using the pending_queue,
we need to make sure it is accounted for properly.

darwin,process: feed kevent the signal to reap children (#3893)

Since we are emulating this event, but are not using the pending_queue,
we need to make sure it is accounted for properly. Also DRY some of the
reset_timeout code here.

This was observed to cause a hang in certain rare cases, particularly on
M1 machines.

Do a bit of code cleanup too, since we do not need to initialize the
internal signal handling pipe if it will not be used.

show more ...

src: add new metrics APIs (#3749)

The following metrics are now always recorded and available via the new
uv_metrics_info() API.

* loop_count: Number of event loop iterations.

src: add new metrics APIs (#3749)

The following metrics are now always recorded and available via the new
uv_metrics_info() API.

* loop_count: Number of event loop iterations.
* events: Total number of events processed by the event handler.
* events_waiting: Total number of events waiting in the event queue when
the event provider request was made.

Benchmarking has shown no noticeable impact recording these metrics.

PR-URL: https://github.com/libuv/libuv/pull/3749

show more ...

build: add MemorySanitizer (MSAN) support (#3788)

- unpoison results from linux system call wrappers

- unpoison results from stat/fstat/lstat to pacify clang 14
(fixed in late

build: add MemorySanitizer (MSAN) support (#3788)

- unpoison results from linux system call wrappers

- unpoison results from stat/fstat/lstat to pacify clang 14
(fixed in later versions)

- add MSAN build option

- turn on MSAN CI build

show more ...

kqueue: DRY file descriptor deletion logic (#3746)

Remove the TODO that says to batch up kevent(EV_DELETE) system calls.
It's unduly complicated in case of errors.

If you pass a

kqueue: DRY file descriptor deletion logic (#3746)

Remove the TODO that says to batch up kevent(EV_DELETE) system calls.
It's unduly complicated in case of errors.

If you pass an out array to kevent(), it will store error records that
tell you which file descriptor caused the error but it also stores new
events in the array, complicating state management.

If you don't pass an out array, it simply doesn't tell you anything
except that _something_ failed. Optimistically trying batch deletion
and falling back to one-by-one deletion is probably possible but is
something of a deoptimization and also feels somewhat dangerous.

macOS has a mostly-undocumented kevent_qos() system call that accepts a
KEVENT_FLAG_ERROR_EVENTS flag that fixes the first problem but that of
course isn't portable.

Long story short, it seems like too much hassle for too little payoff.
Libuv has been doing one-by-one deletion for over a decade now and no
one complained about performance so far so let's just stick with that.

show more ...

unix: don't accept() connections in a loop (#3696)

After analysis of many real-world programs I've come to conclude that
accepting in a loop is nearly always suboptimal.

1. 99.9

unix: don't accept() connections in a loop (#3696)

After analysis of many real-world programs I've come to conclude that
accepting in a loop is nearly always suboptimal.

1. 99.9% of the time the second accept() call fails with EAGAIN, meaning
there are no additional connections to accept. Not super expensive
in isolation but it adds up.

2. When there are more connections to accept but the listen socket is
shared between multiple processes (ex. the Node.js cluster module),
libuv's greedy behavior necessitated the UV_TCP_SINGLE_ACCEPT hack
to slow it down in order to give other processes a chance.

Accepting a single connection and relying on level-triggered polling to
get notified on the next incoming connection both simplifies the code
and optimizes for the common case.

show more ...

kqueue: skip EVFILT_PROC when invalidating fds (#3629)

On NetBSD with libuv 1.44.1 we see that cmake occasionally hangs
waiting for a child process to exit, with libuv waiting forever fo

kqueue: skip EVFILT_PROC when invalidating fds (#3629)

On NetBSD with libuv 1.44.1 we see that cmake occasionally hangs
waiting for a child process to exit, with libuv waiting forever for
`kevent` to deliver more events that never come. The child process has
already exited and is waiting to be collected with `waitpid`.

The hang occurs when the batch of events returned by one call to
`kevent` contains both a EVFILT_READ event for an fd and a later
EVFILT_PROC record for the PID with the same value as the earlier fd.
What happens is that `uv__platform_invalidate_fd` is called to
invalidate events later in the same batch for the fd, but
`uv__platform_invalidate_fd` invalidates the later EVFILT_PROC event
too because it sees the same "ident" value and does not check the
"filter" value to differentiate "ident" values that refer to fds vs.
"ident" values that refer to PIDs.

Add a check for the "filter" value to avoid confusing these two
different kinds of event "ident" values.

show more ...

process: fix hang after NOTE_EXIT (#3521)

Bug #3504 seems to affect more platforms than just OpenBSD. As this
seems to be a race condition in these kernels, we do not want to fail
be

process: fix hang after NOTE_EXIT (#3521)

Bug #3504 seems to affect more platforms than just OpenBSD. As this
seems to be a race condition in these kernels, we do not want to fail
because of it. Instead, we remove the WNOHANG flag from waitpid, and
track exactly which processes have exited. Should also be a slight speed
improvement for excessively large numbers of live children.

show more ...

process,bsd: handle kevent NOTE_EXIT failure (#3451)

The kernel may return ESRCH if the child has already exited here.
This is rather annoying, and means we must indirectly handle
no

process,bsd: handle kevent NOTE_EXIT failure (#3451)

The kernel may return ESRCH if the child has already exited here.
This is rather annoying, and means we must indirectly handle
notification to our event loop of the process exit.

Refs: https://github.com/libuv/libuv/pull/3441
Refs: https://github.com/libuv/libuv/pull/3257

show more ...

process: monitor for exit with kqueue on BSDs (#3441)

This adds a workaround for an xnu kernel bug that sometimes results in
SIGCHLD not being delivered. The workaround is to use kevent

process: monitor for exit with kqueue on BSDs (#3441)

This adds a workaround for an xnu kernel bug that sometimes results in
SIGCHLD not being delivered. The workaround is to use kevent to listen
for EVFILT_PROC/NOTE_EXIT events instead of relying on SIGCHLD on *BSD.

Apple rdar: FB9529664
Refs: https://github.com/libuv/libuv/pull/3257

show more ...

kqueue: ignore write-end closed notifications (#3277)

The combination EVFILT_WRITE+EV_EOF is valid and indicates that the
reader has disconnected (called `shutdown(SHUT_RD)`). Probably v

kqueue: ignore write-end closed notifications (#3277)

The combination EVFILT_WRITE+EV_EOF is valid and indicates that the
reader has disconnected (called `shutdown(SHUT_RD)`). Probably very rare
in practice, but this seems more correct.

Issue noticed (for UV_DISCONNECT clients of uv_poll), while doing
research for https://github.com/libuv/libuv/pull/3250.

show more ...

core: add API to measure event loop idle time

The API addition `uv_metrics_idle_time()` is a thread safe call that
allows the user to retrieve the amount of time the event loop has spent

core: add API to measure event loop idle time

The API addition `uv_metrics_idle_time()` is a thread safe call that
allows the user to retrieve the amount of time the event loop has spent
in the kernel's event provider (i.e. poll). It was done this way to
allow retrieving this value without needing to interrupt the execution
of the event loop. This option can be enabled by passing
`UV_METRICS_IDLE_TIME` to `uv_loop_configure()`.

One important aspect of this change is, when enabled, to always first
call the event provider with a `timeout == 0`. This allows libuv to know
whether any events were waiting in the event queue when the event
provider was called. The importance of this is because libuv is tracking
the amount of "idle time", not "poll time". Thus the provider entry time
is not recorded when `timeout == 0` (the event provider never idles in
this case).

While this does add a small amount of overhead, when enabled, but the
overhead decreases when the event loop has a heavier load. This is
because poll events will be waiting when the event provider is called.
Thus never actually recording the provider entry time.

Checking if `uv_loop_t` is configured with `UV_METRICS_IDLE_TIME` always
happens in `uv__metrics_set_provider_entry_time()` and
`uv__metrics_update_idle_time()`. Making the conditional logic wrapping
each call simpler and allows for instrumentation to always hook into
those two function calls.

Rather than placing the fields directly on `uv__loop_internal_fields_t`
add the struct `uv__loop_metrics_t` as a location for future metrics API
additions.

Tests and additional documentation has been included.

PR-URL: https://github.com/libuv/libuv/pull/2725
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Jameson Nash <vtjnash@gmail.com>

show more ...

unix: use relaxed loads/stores for feature checks

Make ThreadSanitizer stop complaining about the static variables that
libuv uses to record the presence (or lack) of system calls and ot

unix: use relaxed loads/stores for feature checks

Make ThreadSanitizer stop complaining about the static variables that
libuv uses to record the presence (or lack) of system calls and other
kernel features.

Fixes: https://github.com/libuv/libuv/issues/2884
PR-URL: https://github.com/libuv/libuv/pull/2886
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jameson Nash <vtjnash@gmail.com>

show more ...

fsevents: stop using fsevents to watch files

Goes back to just using it to watch folders,
but keeps the other logic changes around.

Refs: https://github.com/libuv/libuv/pull/387

fsevents: stop using fsevents to watch files

Goes back to just using it to watch folders,
but keeps the other logic changes around.

Refs: https://github.com/libuv/libuv/pull/387
Refs: https://github.com/libuv/libuv/pull/2082
Refs: https://github.com/libuv/libuv/pull/1572
Refs: https://github.com/nodejs/node/issues/29460
Fixes: https://github.com/libuv/libuv/issues/2488
Closes: https://github.com/libuv/libuv/pull/2452
PR-URL: https://github.com/libuv/libuv/pull/2459
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <s@saghul.net>

show more ...

unix: fall back to kqueue on older macOS systems

Newer macOS systems (10.7+) use fsevents() for the fs_event backend,
but older macOS systems don't have full functionality for this in

unix: fall back to kqueue on older macOS systems

Newer macOS systems (10.7+) use fsevents() for the fs_event backend,
but older macOS systems don't have full functionality for this in
the macOS fsevents() API. Fall back to kqueue for these systems.

PR-URL: https://github.com/libuv/libuv/pull/2290
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

show more ...

bsd: plug uv_fs_event_start() error path fd leak

Close the watched file's file descriptor when initialization fails due
to out-of-memory conditions.

PR-URL: https://github.com/l

bsd: plug uv_fs_event_start() error path fd leak

Close the watched file's file descriptor when initialization fails due
to out-of-memory conditions.

PR-URL: https://github.com/libuv/libuv/pull/2197
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...