unix,win: remove unused req parameter from macros (#4435)

Remove the unused `req` parameter from the uv__req_register and
uv__req_unregister macros.

unix,win: replace QUEUE with struct uv__queue (#4022)

Recent versions of gcc have started emitting warnings about the liberal
type casting inside the QUEUE macros. Although the warnings

unix,win: replace QUEUE with struct uv__queue (#4022)

Recent versions of gcc have started emitting warnings about the liberal
type casting inside the QUEUE macros. Although the warnings are false
positives, let's use them as the impetus to switch to a type-safer and
arguably cleaner approach.

Fixes: https://github.com/libuv/libuv/issues/4019

show more ...

src: fix events/events_waiting metrics counter (#3957)

The worker pool calls all callbacks locally within the queue. So the
value of nevents doesn't properly reflect that case. Increase

src: fix events/events_waiting metrics counter (#3957)

The worker pool calls all callbacks locally within the queue. So the
value of nevents doesn't properly reflect that case. Increase the number
of events directly from the worker pool's callback to correct this.

In order to properly determine if the events_waiting counter needs to be
incremented, store the timeout value at the time the event provider was
called.

show more ...

linux: introduce io_uring support (#3952)

Add io_uring support for several asynchronous file operations:

- read, write
- fsync, fdatasync
- stat, fstat, lstat

io_ur

linux: introduce io_uring support (#3952)

Add io_uring support for several asynchronous file operations:

- read, write
- fsync, fdatasync
- stat, fstat, lstat

io_uring is used when the kernel is new enough, otherwise libuv simply
falls back to the thread pool.

Performance looks great; an 8x increase in throughput has been observed.

This work was sponsored by ISC, the Internet Systems Consortium.

Fixes: https://github.com/libuv/libuv/issues/1947

show more ...

unix,win: give thread pool threads an 8 MB stack (#3787)

Give the threads in the thread pool a stack size that is consistent
across platforms and architectures.

Fixes: https://g

unix,win: give thread pool threads an 8 MB stack (#3787)

Give the threads in the thread pool a stack size that is consistent
across platforms and architectures.

Fixes: https://github.com/libuv/libuv/issues/3786

show more ...

zos: use destructor for uv__threadpool_cleanup() (#3376)

On z/OS, instead of calling the uv__threadpool_cleanup() function from
inside uv_library_shutdown(), the destructor attribute mus

zos: use destructor for uv__threadpool_cleanup() (#3376)

On z/OS, instead of calling the uv__threadpool_cleanup() function from
inside uv_library_shutdown(), the destructor attribute must be used;
otherwise, tests will fail with exit code 1 and no output. Additionally,
post() does not need to be called when the destructor attribute is used.

Also adds uv__os390_cleanup() function to clean System V message queue
on z/OS.

Co-authored-by: Igor Todorovski <itodorov@ca.ibm.com>
Co-authored-by: Gaby Baghdadi <baghdadi@ca.ibm.com>

show more ...

cleanup,win: Remove _WIN32 guards on threadpool

Fixes: https://github.com/libuv/libuv/issues/2980
Refs: https://github.com/nodejs/node/pull/35021
Reviewed-By: Ben Noordhuis <info@bno

cleanup,win: Remove _WIN32 guards on threadpool

Fixes: https://github.com/libuv/libuv/issues/2980
Refs: https://github.com/nodejs/node/pull/35021
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Jameson Nash <vtjnash@gmail.com>

show more ...

unix,win: add uv_library_shutdown()

Make it possible to explicitly tell libuv to release any resources
it's still holding onto (memory, threads, file descriptors, etc.)

Before t

unix,win: add uv_library_shutdown()

Make it possible to explicitly tell libuv to release any resources
it's still holding onto (memory, threads, file descriptors, etc.)

Before this commit, cleanup was performed in various destructors.
This commit centralizes the cleanup logic, enabling the addition of
`uv_library_shutdown()`, but maintains the current observable behavior
of cleaning up when libuv is unloaded by means of `dlclose(3)`.

Fixes: https://github.com/libuv/libuv/issues/2763
PR-URL: https://github.com/libuv/libuv/pull/2764
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix,win: add uv_random()

Add an API for obtaining cryptographically strong random data from the
system PRNG.

Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
Refs: https

unix,win: add uv_random()

Add an API for obtaining cryptographically strong random data from the
system PRNG.

Co-authored-by: Saúl Ibarra Corretgé <s@saghul.net>
Refs: https://github.com/libuv/libuv/pull/1055
PR-URL: https://github.com/libuv/libuv/pull/2347
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

show more ...

threadpool: increase UV_THREADPOOL_SIZE limit

Increase the UV_THREADPOOL_SIZE limit to 1024 and update the docs.

Fixes: https://github.com/libuv/libuv/pull/2296
PR-URL: https://

threadpool: increase UV_THREADPOOL_SIZE limit

Increase the UV_THREADPOOL_SIZE limit to 1024 and update the docs.

Fixes: https://github.com/libuv/libuv/pull/2296
PR-URL: https://github.com/libuv/libuv/pull/2314
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix,win: fix threadpool race condition

90891b4232e91dbd7a2e2077e4d23d16a374b41d introduced a race
condition when accessing `slow_io_work_running` – it is being
increased and later d

unix,win: fix threadpool race condition

90891b4232e91dbd7a2e2077e4d23d16a374b41d introduced a race
condition when accessing `slow_io_work_running` – it is being
increased and later decreased as part of the worker thread loop,
but was accessed with different mutexes during these operations.

This fixes the race condition by making sure both accesses
are protected through the global `mutex` of `threadpool.c`.

This fixes a number of flaky Node.js tests.

Refs: https://github.com/libuv/libuv/pull/1845
Refs: https://github.com/nodejs/reliability/issues/18
Refs: https://github.com/nodejs/node/issues/23089
Refs: https://github.com/nodejs/node/issues/23067
Refs: https://github.com/nodejs/node/issues/23066
Refs: https://github.com/nodejs/node/issues/23219
PR-URL: https://github.com/libuv/libuv/pull/2021
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

show more ...

unix,win: limit concurrent DNS calls to nthreads/2

If `nthreads / 2` (rounded up) DNS calls are outstanding,
queue more work of that kind instead of letting it take over
more positio

unix,win: limit concurrent DNS calls to nthreads/2

If `nthreads / 2` (rounded up) DNS calls are outstanding,
queue more work of that kind instead of letting it take over
more positions in the thread pool, blocking other work
such as the (usually much faster) file system I/O or
user-scheduled work.

Fixes: https://github.com/nodejs/node/issues/8436
PR-URL: https://github.com/libuv/libuv/pull/1845
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

show more ...

unix,win: wait for threads to start

It appears that, at least on Windows, the process terminates abnormally
when the program exits before the worker threads complete initializing.
Wa

unix,win: wait for threads to start

It appears that, at least on Windows, the process terminates abnormally
when the program exits before the worker threads complete initializing.
Wait for the threads to spin up in `init_threads()` to avoid that.

PR-URL: https://github.com/libuv/libuv/pull/1662
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Jamie Davis <davisjam@vt.edu>

show more ...

Revert "unix,win: wait for threads to start"

Reverted for suspected regressions in the Node.js test suite on Windows.

Also revert e99ac4c1f4 ("test: add threadpool init/teardown tes

Revert "unix,win: wait for threads to start"

Reverted for suspected regressions in the Node.js test suite on Windows.

Also revert e99ac4c1f4 ("test: add threadpool init/teardown test") since
it depends on 5486f6bd51.

This reverts commit 5486f6bd517382284109ff2f5355b61de417c4e8.
This reverts commit e99ac4c1f4eb3b049a9e147cb53a666506858121.

PR-URL: https://github.com/libuv/libuv/pull/1656
Refs: https://github.com/libuv/libuv/pull/1639
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>

show more ...

unix,win: wait for threads to start

It appears that, at least on Windows, the process terminates abnormally
when the program exits before the worker threads complete initializing.
Wa

unix,win: wait for threads to start

It appears that, at least on Windows, the process terminates abnormally
when the program exits before the worker threads complete initializing.
Wait for the threads to spin up in `init_threads()` to avoid that.

Refs: https://github.com/libuv/libuv/pull/1613#issuecomment-344958863
PR-URL: https://github.com/libuv/libuv/pull/1639
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

show more ...

unix,windows: refactor request init logic

Fixes a TODO in src/threadpool.c. Updates the Windows code to drop the
unused `loop` parameter in calls to uv_req_init().

PR-URL: http

unix,windows: refactor request init logic

Fixes a TODO in src/threadpool.c. Updates the Windows code to drop the
unused `loop` parameter in calls to uv_req_init().

PR-URL: https://github.com/libuv/libuv/pull/1091
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

unix: make loops and watchers usable after fork()

Added the uv_loop_fork() API that must be called in a child process to
continue using an existing loop. Internally this calls a uv__io_f

unix: make loops and watchers usable after fork()

Added the uv_loop_fork() API that must be called in a child process to
continue using an existing loop. Internally this calls a uv__io_fork
function for each supported platform, similar to the way
uv__platform_loop_init works.

After this call, existing and new IO, async and signal watchers will
contiue working as before on all platforms, as will the
threadpool (although any threads it was using are of course gone).

On Linux and BSDs that use kqueue, existing and new fsevent watchers
will also continue to work as expected. On OS X, though, directory
fsevents will not be able to use the optimized CoreFoundation path if
they had already been used in the parent process, instead falling back
to the kqueue path used on other BSDs.

Existing fsevent watchers will not function on AIX or SunOS. This
could be relatively easily fixed by someone with AIX knowledge in the
future, but SunOS will require some additional work to keep track if
the watchers.

A new test file, test/test-fork.c, was added to contain fork-related
tests to verify functionality in the child process.

PR-URL: https://github.com/libuv/libuv/pull/846
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

src: replace QUEUE_SPLIT with QUEUE_MOVE

All uses of QUEUE_SPLIT in libuv split the list at the head so introduce
a QUEUE_MOVE macro that automates that.

PR-URL: https://github.

src: replace QUEUE_SPLIT with QUEUE_MOVE

All uses of QUEUE_SPLIT in libuv split the list at the head so introduce
a QUEUE_MOVE macro that automates that.

PR-URL: https://github.com/libuv/libuv/pull/565
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

threadpool: fix thread starvation bug

Commit 0f1bdb6 ("threadpool: send signal only when queue is empty")
introduces a regression where work is not evenly distributed across
the thre

threadpool: fix thread starvation bug

Commit 0f1bdb6 ("threadpool: send signal only when queue is empty")
introduces a regression where work is not evenly distributed across
the thread pool because the work queue's condition variable is only
signalled when the queue is empty, even when there are waiting workers.

It doesn't turn into outright deadlock because there is always
at least one thread making forward progress but it does degrade
throughput, sometimes massively so.

Signalling whenever there are waiting workers fixes the throughput
issue while still keeping the number of uv_cond_signal() calls low,
which was the motivation for commit 0f1bdb6.

Fixes: https://github.com/libuv/libuv/pull/490
Fixes: https://github.com/libuv/libuv/issues/492
PR-URL: https://github.com/libuv/libuv/pull/493
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

threadpool: send signal only when queue is empty

PR-URL: https://github.com/libuv/libuv/pull/460
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <sa

threadpool: send signal only when queue is empty

PR-URL: https://github.com/libuv/libuv/pull/460
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

core: add ability to customize memory allocator

This patch is composed by the work done in
https://github.com/libuv/libuv/pull/231 and
https://github.com/libuv/libuv/pull/287 plus so

core: add ability to customize memory allocator

This patch is composed by the work done in
https://github.com/libuv/libuv/pull/231 and
https://github.com/libuv/libuv/pull/287 plus some changes by yours
truly.

Thanks @beevik and @mattsta for their work on this!

PR-URL: https://github.com/libuv/libuv/pull/368
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

show more ...

Revert "memory: add uv_replace_allocator"

This reverts commit c272f1f1bc0bda625e6441d798c110b4064a6ce2.

The concept will come back to libuv, but it needs some more work.

memory: add uv_replace_allocator

With uv_replace_allocator, it's possible to override the default
memory allocator's malloc and free calls with functions of the user's
choosing. This

memory: add uv_replace_allocator

With uv_replace_allocator, it's possible to override the default
memory allocator's malloc and free calls with functions of the user's
choosing. This allows libuv to interoperate with projects requiring a
custom memory allocator.

Internally, all calls to malloc and free have been replaced with
uv__malloc and uv__free, respectively. The uv__malloc and uv__free
functions call malloc and free unless they have been overridden by a
previous call to uv_replace_allocator.

As part of this change, the special aligned memory allocations
performed in src/win/fs-event.c have been replaced with standard
allocations. The 4-byte alignment being requested in this file was
unnecessary, since standard allocators already guarantee at least an
8-byte alignment.

PR-URL: https://github.com/libuv/libuv/pull/231
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com>

show more ...

windows: don't use atexit for cleaning up the threadpool

If libuv is loaded as a DLL and is later unloaded deadlocks can happen
when running atexit handlers, so we can't use synchronizat

windows: don't use atexit for cleaning up the threadpool

If libuv is loaded as a DLL and is later unloaded deadlocks can happen
when running atexit handlers, so we can't use synchronization
priomitives or join threads there.

For reference see https://github.com/saghul/pyuv/issues/171

show more ...

unix, windows: use the same threadpool implementation