low-speed-limit: avoid timeout flood

Introducing Curl_expire_latest(). To be used when we the code flow only
wants to get called at a later time that is "no later than X" so that
som

low-speed-limit: avoid timeout flood

Introducing Curl_expire_latest(). To be used when we the code flow only
wants to get called at a later time that is "no later than X" so that
something can be checked (and another timeout be added).

The low-speed logic for example could easily be made to set very many
expire timeouts if it would be called faster or sooner than what it had
set its own timer and this goes for a few other timers too that aren't
explictiy checked for timer expiration in the code.

If there's no condition the code that says if(time-passed >= TIME), then
Curl_expire_latest() is preferred to Curl_expire().

If there exists such a condition, it is on the other hand important that
Curl_expire() is used and not the other.

Bug: http://curl.haxx.se/mail/lib-2014-06/0235.html
Reported-by: Florian Weimer

show more ...

multi: remove MULTI_TIMEOUT_INACCURACY

With the recently added timeout "reminder" functionality, there's no
reason left for us to execute timeout code before the time is
ripe. Simpli

multi: remove MULTI_TIMEOUT_INACCURACY

With the recently added timeout "reminder" functionality, there's no
reason left for us to execute timeout code before the time is
ripe. Simplifies the handling too.

This will make the *TIMEOUT and *CONNECTTIMEOUT options more accurate
again, which probably is most important when the *_MS versions are used.

In multi_socket, make sure to update 'now' after having handled activity
on a socket.

show more ...

multi: add timer inaccuracy margin to timeout/connecttimeout

Since all systems have inaccuracy in the timeout handling it is
imperative that we add an inaccuracy margin to the general ti

multi: add timer inaccuracy margin to timeout/connecttimeout

Since all systems have inaccuracy in the timeout handling it is
imperative that we add an inaccuracy margin to the general timeout and
connecttimeout handling with the multi interface. This way, when the
timeout fires we should be fairly sure that it has passed the timeout
value and will be suitably detected.

For cases where the timeout fire before the actual timeout, we would
otherwise consume the timeout action and still not run the timeout code
since the condition wasn't met.

Reported-by: He Qin
Bug: http://curl.haxx.se/bug/view.cgi?id=1298

show more ...

multi_socket: improved 100-continue timeout handling

When waiting for a 100-continue response from the server, the
Curl_readwrite() will refuse to run if called until the timeout has bee

multi_socket: improved 100-continue timeout handling

When waiting for a 100-continue response from the server, the
Curl_readwrite() will refuse to run if called until the timeout has been
reached.

We timeout code in multi_socket() allows code to run slightly before the
actual timeout time, so for test 154 it could lead to the function being
executed but refused in Curl_readwrite() and then the application would
just sit idling forever.

This was detected with runtests.pl -e on test 154.

show more ...

multi_socket: react on socket close immediately

As a remedy to the problem when a socket gets closed and a new one is
opened with the same file descriptor number and as a result
mult

multi_socket: react on socket close immediately

As a remedy to the problem when a socket gets closed and a new one is
opened with the same file descriptor number and as a result
multi.c:singlesocket() doesn't detect the difference, the new function
Curl_multi_closed() gets told when a socket is closed so that it can be
removed from the socket hash. When the old one has been removed, a new
socket should be detected fine by the singlesocket() on next invoke.

Bug: http://curl.haxx.se/bug/view.cgi?id=1248
Reported-by: Erik Johansson

show more ...

curl_easy_init: use less mallocs

By introducing an internal alternative to curl_multi_init() that accepts
parameters to set the hash sizes, easy handles will now use tiny socket
and

curl_easy_init: use less mallocs

By introducing an internal alternative to curl_multi_init() that accepts
parameters to set the hash sizes, easy handles will now use tiny socket
and connection hash tables since it will only ever add a single easy
handle to that multi handle.

This decreased the number mallocs in test 40 (which is a rather simple
and typical easy interface use case) from 1142 to 138. The maximum
amount of memory allocated used went down from 118969 to 78805.

show more ...

Multiple pipelines and limiting the number of connections.

Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
opt

Multiple pipelines and limiting the number of connections.

Introducing a number of options to the multi interface that
allows for multiple pipelines to the same host, in order to
optimize the balance between the penalty for opening new
connections and the potential pipelining latency.

Two new options for limiting the number of connections:

CURLMOPT_MAX_HOST_CONNECTIONS - Limits the number of running connections
to the same host. When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished, so we can reuse the connection.

CURLMOPT_MAX_TOTAL_CONNECTIONS - Limits the number of connections in total.
When adding a handle that exceeds this limit,
that handle will be put in a pending state until another handle is
finished. The free connection will then be reused, if possible, or
closed if the pending handle can't reuse it.

Several new options for pipelining:

CURLMOPT_MAX_PIPELINE_LENGTH - Limits the pipeling length. If a
pipeline is "full" when a connection is to be reused, a new connection
will be opened if the CURLMOPT_MAX_xxx_CONNECTIONS limits allow it.
If not, the handle will be put in a pending state until a connection is
ready (either free or a pipe got shorter).

CURLMOPT_CONTENT_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a transfer with a content
length that is larger than this.

CURLMOPT_CHUNK_LENGTH_PENALTY_SIZE - A pipelined connection will not
be reused if it is currently processing a chunk larger than this.

CURLMOPT_PIPELINING_SITE_BL - A blacklist of hosts that don't allow
pipelining.

CURLMOPT_PIPELINING_SERVER_BL - A blacklist of server types that don't allow
pipelining.

See the curl_multi_setopt() man page for details.

show more ...

Revert changes relative to lib/*.[ch] recent renaming

This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:

f871de0... build: make use

Revert changes relative to lib/*.[ch] recent renaming

This reverts renaming and usage of lib/*.h header files done
28-12-2012, reverting 2 commits:

f871de0... build: make use of 76 lib/*.h renamed files
ffd8e12... build: rename 76 lib/*.h files

This also reverts removal of redundant include guard (redundant thanks
to changes in above commits) done 2-12-2013, reverting 1 commit:

c087374... curl_setup.h: remove redundant include guard

This also reverts renaming and usage of lib/*.c source files done
3-12-2013, reverting 3 commits:

13606bb... build: make use of 93 lib/*.c renamed files
5b6e792... build: rename 93 lib/*.c files
7d83dff... build: commit 13606bbfde follow-up 1

Start of related discussion thread:

http://curl.haxx.se/mail/lib-2013-01/0012.html

Asking for confirmation on pushing this revertion commit:

http://curl.haxx.se/mail/lib-2013-01/0048.html

Confirmation summary:

http://curl.haxx.se/mail/lib-2013-01/0079.html

NOTICE: The list of 2 files that have been modified by other
intermixed commits, while renamed, and also by at least one
of the 6 commits this one reverts follows below. These 2 files
will exhibit a hole in history unless git's '--follow' option
is used when viewing logs.

lib/curl_imap.h
lib/curl_smtp.h

show more ...

lib/*.h: use our standard naming scheme for header inclusion guards

multi: support timeouts

Curl_expire() is now expanded to hold a list of timeouts for each easy
handle. Only the closest in time will be the one used as the primary
timeout for the ha

multi: support timeouts

Curl_expire() is now expanded to hold a list of timeouts for each easy
handle. Only the closest in time will be the one used as the primary
timeout for the handle and will be used for the splay tree (which sorts
and lists all handles within the multi handle).

When the main timeout has triggered/expired, the next timeout in time
that is kept in the list will be moved to the main timeout position and
used as the key to splay with. This way, all timeouts that are set with
Curl_expire() internally will end up as a proper timeout. Previously any
Curl_expire() that set a _later_ timeout than what was already set was
just silently ignored and thus missed.

Setting Curl_expire() with timeout 0 (zero) will cancel all previously
added timeouts.

Corrects known bug #62.

show more ...

remove the CVSish $Id$ lines

Adjusted to take in account that...

With the curl memory tracking feature decoupled from the debug build feature,
CURLDEBUG and DEBUGBUILD preprocessor symbol definitions are used as fol

Adjusted to take in account that...

With the curl memory tracking feature decoupled from the debug build feature,
CURLDEBUG and DEBUGBUILD preprocessor symbol definitions are used as follows:

CURLDEBUG used for curl debug memory tracking specific code (--enable-curldebug)

DEBUGBUILD used for debug enabled specific code (--enable-debug)

show more ...

fix compiler warning: defined but not used

Fixed some minor mismatched types found by splint.

Ravi Pratap provided fixes for HTTP pipelining

Reported in #1561470 (http://curl.haxx.se/bug/view.cgi?id=1561470), libcurl
would crash if a bad function sequence was used when shutting down after
using the multi interface (i.e using easy_

Reported in #1561470 (http://curl.haxx.se/bug/view.cgi?id=1561470), libcurl
would crash if a bad function sequence was used when shutting down after
using the multi interface (i.e using easy_cleanup after multi_cleanup) so
precautions have been added to make sure it doesn't any more - test case 529
was added to verify.

show more ...

Major overhaul introducing http pipelining support and shared connection
cache within the multi handle.

First curl_multi_socket() commit. Should primarily be considered as an internal
code rearrange to fit the future better.

Updated the copyright year since changes have been this year.

FTP code turned into state machine. Not completely yet, but a good start.
The tag 'before_ftp_statemachine' was set just before this commit in case
of future need.

Renamed easy.h and multi.h to easyif.h and multiif.h to make sure they don't
shadow our public headers with the former names.