Revert "linux: eliminate a read on eventfd per wakeup (#4400)" (#4585)

This reverts commit e5cb1d3d3d4ab3178ac567fb6a7f0f4b5eef3083.

Reason: bisecting says it breaks dnstap.

Revert "linux: eliminate a read on eventfd per wakeup (#4400)" (#4585)

This reverts commit e5cb1d3d3d4ab3178ac567fb6a7f0f4b5eef3083.

Reason: bisecting says it breaks dnstap.

Also revert commit 27134547ff ("kqueue: use EVFILT_USER for async if
available") because otherwise the first commit doesn't revert cleanly,
with enough conflicts in src/unix/async.c that I'm not comfortable
fixing those up manually.

Fixes: https://github.com/libuv/libuv/issues/4584

show more ...

kqueue: use EVFILT_USER for async if available

Establishes a user event for kqueue to eliminate the overhead
of the pipe and the system call read(2) per wakeup event.

---------

kqueue: use EVFILT_USER for async if available

Establishes a user event for kqueue to eliminate the overhead
of the pipe and the system call read(2) per wakeup event.

---------

Signed-off-by: Andy Pan <i@andypan.me>

Co-authored-by: Jameson Nash <vtjnash@gmail.com>

show more ...

linux: eliminate a read on eventfd per wakeup (#4400)

Register the eventfd with EPOLLET to enable edge-triggered notification
where we're able to eliminate the overhead of reading the ev

linux: eliminate a read on eventfd per wakeup (#4400)

Register the eventfd with EPOLLET to enable edge-triggered notification
where we're able to eliminate the overhead of reading the eventfd via
system call on each wakeup event.

When the eventfd counter reaches the maximum value of the unsigned 64-bit,
which may not happen for the entire lifetime of the process, we rewind the
counter and retry.

This optimization saves one system call on each event-loop wakeup,
eliminating the overhead of read(2) as well as the extra latency
for each epoll wakeup.

show more ...

unix,win: replace QUEUE with struct uv__queue (#4022)

Recent versions of gcc have started emitting warnings about the liberal
type casting inside the QUEUE macros. Although the warnings

unix,win: replace QUEUE with struct uv__queue (#4022)

Recent versions of gcc have started emitting warnings about the liberal
type casting inside the QUEUE macros. Although the warnings are false
positives, let's use them as the impetus to switch to a type-safer and
arguably cleaner approach.

Fixes: https://github.com/libuv/libuv/issues/4019

show more ...

unix: remove busy loop from uv_async_send (#3879)

The current fix (libuv#2231) was found to be slow in certain cases. This
change should improve scalabaility a bit by only incurring the

unix: remove busy loop from uv_async_send (#3879)

The current fix (libuv#2231) was found to be slow in certain cases. This
change should improve scalabaility a bit by only incurring the spin loop
delay while closing an UV_ASYNC. It also is intended to slightly improve
the behavior after uv_loop_close is called, by parking all of the
pending flags as set, so that it will not access the loop at all (until
the uv_async_t memory is freed, which we leave still to the
responsibility of the user).

Note that this bug appears to still exist on Win32, though it's harder
to address without the refactoring done to this code on libuv master.

Takes some inspiration from https://github.com/libuv/libuv/pull/2654
Takes some inspiration from https://github.com/libuv/libuv/pull/2656
Refs: https://github.com/libuv/libuv/pull/2231

show more ...

unix: switch to c11 atomics (#3688)

Fixes: https://github.com/libuv/libuv/issues/3683

stream: add uv_pipe and uv_socketpair to the API

Equivalents of `pipe` and `socketpair` for cross-platform use.

PR-URL: https://github.com/libuv/libuv/pull/2953
Reviewed-By: San

stream: add uv_pipe and uv_socketpair to the API

Equivalents of `pipe` and `socketpair` for cross-platform use.

PR-URL: https://github.com/libuv/libuv/pull/2953
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

show more ...

unix: yield cpu when spinlocking on async handle

Fixes: https://github.com/libuv/libuv/issues/2769
PR-URL: https://github.com/libuv/libuv/pull/2772
Reviewed-By: Anna Henningsen <anna

unix: yield cpu when spinlocking on async handle

Fixes: https://github.com/libuv/libuv/issues/2769
PR-URL: https://github.com/libuv/libuv/pull/2772
Reviewed-By: Anna Henningsen <anna@addaleax.net>

show more ...

linux: simplify uv__async_eventfd()

Assume the presence of the eventfd2() system call on Linux. It was added
in 2.6.27 and our baseline is 2.6.32.

PR-URL: https://github.com/lib

linux: simplify uv__async_eventfd()

Assume the presence of the eventfd2() system call on Linux. It was added
in 2.6.27 and our baseline is 2.6.32.

PR-URL: https://github.com/libuv/libuv/pull/2665
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix: fix race condition in uv_async_send()

After inspection of the code, it seems like there is a race window
between the cmpxchgi() and uv__async_send() calls.

If the event lo

unix: fix race condition in uv_async_send()

After inspection of the code, it seems like there is a race window
between the cmpxchgi() and uv__async_send() calls.

If the event loop thread is already busy looping over the async handles,
it can invoke the callback - which in turn can close the handle - before
the other thread reaches the uv__async_send() call. That's bad because
it accesses the handle that at that point might not be valid anymore.

Fix that by introducing an ad hoc spinlock that blocks the event loop
thread until the sending thread is done. It's not pretty or elegant
but it fixes the immediate bug and appears to have no measurable
impact on the async handle benchmarks.

Fixes: https://github.com/libuv/libuv/issues/2226
PR-URL: https://github.com/libuv/libuv/pull/2231
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

include,src: introduce UV__ERR() macro

Using -errno, -E**, and -pthread_function() can be
error prone, and breaks compatibility with some operating
systems that already negate errno'

include,src: introduce UV__ERR() macro

Using -errno, -E**, and -pthread_function() can be
error prone, and breaks compatibility with some operating
systems that already negate errno's (e.g. Haiku).

This commit adds a UV__ERR() macro that ensures libuv
errors are negative.

Fixes: https://github.com/libuv/help/issues/39
PR-URL: https://github.com/libuv/libuv/pull/1687
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

build: add -Wstrict-prototypes

Fixes: https://github.com/libuv/libuv/pull/1320
PR-URL: https://github.com/libuv/libuv/pull/1326
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Revie

build: add -Wstrict-prototypes

Fixes: https://github.com/libuv/libuv/pull/1320
PR-URL: https://github.com/libuv/libuv/pull/1326
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

show more ...

unix: make loops and watchers usable after fork()

Added the uv_loop_fork() API that must be called in a child process to
continue using an existing loop. Internally this calls a uv__io_f

unix: make loops and watchers usable after fork()

Added the uv_loop_fork() API that must be called in a child process to
continue using an existing loop. Internally this calls a uv__io_fork
function for each supported platform, similar to the way
uv__platform_loop_init works.

After this call, existing and new IO, async and signal watchers will
contiue working as before on all platforms, as will the
threadpool (although any threads it was using are of course gone).

On Linux and BSDs that use kqueue, existing and new fsevent watchers
will also continue to work as expected. On OS X, though, directory
fsevents will not be able to use the optimized CoreFoundation path if
they had already been used in the parent process, instead falling back
to the kqueue path used on other BSDs.

Existing fsevent watchers will not function on AIX or SunOS. This
could be relatively easily fixed by someone with AIX knowledge in the
future, but SunOS will require some additional work to keep track if
the watchers.

A new test file, test/test-fork.c, was added to contain fork-related
tests to verify functionality in the child process.

PR-URL: https://github.com/libuv/libuv/pull/846
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix: simplify async watcher dispatch logic

Remove the unused `nevents` parameter from `uv__async_event()`
and remove the indirection of having a separate `uv__async`
type. There is

unix: simplify async watcher dispatch logic

Remove the unused `nevents` parameter from `uv__async_event()`
and remove the indirection of having a separate `uv__async`
type. There is only one instance per event loop these days.

This incidentally removes the `assert(n == sizeof(val))` in a
Linux-specific code path that some users seem to hit from time
to time. The cause is not well-understood and I've never been
able to reproduce it myself. Presumably libuv gets an EAGAIN
when trying to read from the eventfd but when and why that
happens is unclear.

Since the byte count is unused, removing the assert seems safe.
Worst case, libuv sometimes iterates over the async watcher list
when it doesn't have to.

Fixes: https://github.com/libuv/libuv/issues/1171
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>
Supersedes: https://github.com/libuv/libuv/pull/1214

show more ...

unix: use POLL{IN,OUT,etc} constants directly

Remove the UV__POLL defines and use POLL{IN,OUT,etc} directly.
On Linux, we lean on the fact that the POLL constants correspond
one-to-o

unix: use POLL{IN,OUT,etc} constants directly

Remove the UV__POLL defines and use POLL{IN,OUT,etc} directly.
On Linux, we lean on the fact that the POLL constants correspond
one-to-one to their EPOLL counterparts.

Fixes: https://github.com/libuv/libuv/issues/816
PR-URL: https://github.com/libuv/libuv/pull/817
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix: use QUEUE_MOVE when iterating over lists

Replace uses of QUEUE_FOREACH when the list can get modified while
iterating over it, in particular when a callback is made into the
us

unix: use QUEUE_MOVE when iterating over lists

Replace uses of QUEUE_FOREACH when the list can get modified while
iterating over it, in particular when a callback is made into the
user's code. This should fix a number of spurious failures that
people have been reporting.

PR-URL: https://github.com/libuv/libuv/pull/565
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix: fix for uv_async data race

There's a data race in the consuming side of uv_async. The "pending"
flag could be trampled by producing thread causing an async send
event to be mis

unix: fix for uv_async data race

There's a data race in the consuming side of uv_async. The "pending"
flag could be trampled by producing thread causing an async send
event to be missed.

PR-URL: https://github.com/libuv/libuv/pull/189
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix: fix uv__open_cloexec usage

It returns the fd or the negated errno.

linux: reduce file descriptor count of async pipe

Reopen one of the pipe descriptors as read/write through the procfs.
Allows us to close the original pipe file descriptors, saving a fil

linux: reduce file descriptor count of async pipe

Reopen one of the pipe descriptors as read/write through the procfs.
Allows us to close the original pipe file descriptors, saving a file
descriptor on kernels that don't support eventfd(2).

show more ...

unix, windows: removed unused status parameter

async, timer, prepare, idle and check handles don't need the status
parameter.

unix: sanity-check fds before closing

Ensure that close() system calls don't close stdio file descriptors
because that is almost never the intention.

This is also a partial work

unix: sanity-check fds before closing

Ensure that close() system calls don't close stdio file descriptors
because that is almost never the intention.

This is also a partial workaround for a kernel bug that seems to affect
all Linux kernels when stdin is a pipe that gets closed: fd 0 keeps
signalling EPOLLHUP but a subsequent call to epoll_ctl(EPOLL_CTL_DEL)
fails with EBADF. See joyent/node#6271 for details and a test case.

show more ...

unix, windows: allow NULL async callback

Allow a NULL callback so the user doesn't have to provide a dummy when
the actual event is processed by e.g. a check handle callback.

unix, windows: return error codes directly

This commit changes the libuv API to return error codes directly rather
than storing them in a loop-global field.

A code snippet like

unix, windows: return error codes directly

This commit changes the libuv API to return error codes directly rather
than storing them in a loop-global field.

A code snippet like this one:

if (uv_foo(loop) < 0) {
uv_err_t err = uv_last_error(loop);
fprintf(stderr, "%s\n", uv_strerror(err));
}

Should be rewritten like this:

int err = uv_foo(loop);
if (err < 0)
fprintf(stderr, "%s\n", uv_strerror(err));

The rationale for this change is that it should make creating bindings
for other languages a lot easier: dealing with struct return values is
painful with most FFIs and often downright buggy.

show more ...

unix, windows: remove ngx-queue.h

Avoids an extra #include in public headers and stops the ngx_queue_*
types and macros from leaking into user code.

linux: use eventfds for async handles

Use eventfds to drive async handles, fall back to regular pipes on older
kernels (pre-2.6.22).

Gives a nice boost on the async handle bench

linux: use eventfds for async handles

Use eventfds to drive async handles, fall back to regular pipes on older
kernels (pre-2.6.22).

Gives a nice boost on the async handle benchmarks. Before:

12,516,113 async events in 5.0 seconds (2,503,222/s, 1,048,576 unique
async1: 11.95 sec (83,701/sec)
async2: 11.65 sec (85,862/sec)
async4: 5.20 sec (192,154/sec)
async8: 9.97 sec (100,315/sec)
async_pummel_1: 1,000,000 callbacks in 2.56 seconds (389,919/sec)
async_pummel_2: 1,000,000 callbacks in 2.65 seconds (377,205/sec)
async_pummel_4: 1,000,000 callbacks in 2.18 seconds (457,704/sec)
async_pummel_8: 1,000,000 callbacks in 4.19 seconds (238,632/sec)

After:

16,168,081 async events in 5.0 seconds (3,233,616/s, 1,048,576 unique
async1: 11.08 sec (90,213/sec)
async2: 10.17 sec (98,297/sec)
async4: 4.81 sec (207,789/sec)
async8: 8.98 sec (111,419/sec)
async_pummel_1: 1,000,000 callbacks in 1.16 seconds (863,296/sec)
async_pummel_2: 1,000,000 callbacks in 1.45 seconds (691,459/sec)
async_pummel_4: 1,000,000 callbacks in 0.66 seconds (1,514,770/sec)
async_pummel_8: 1,000,000 callbacks in 1.42 seconds (704,549/sec)

That's a speedup from anywhere between 10% to 330%.

show more ...