Abstract the pipe communication into NgxEventConnection, for reuse
by NgxBaseFetch and NgxUrlAsyncFetcher.
Based on Chai's earlier work, but with a few fixes discovered
while working on this and SPDY module compatibility
- Uses less file descriptors, I expect this to be faster but need
measurement is needed to back that.
- Fixed NgxUrlAsyncFetcher actually shutting down its fetchers.
- Fixes a bug where we wouldn't clean idle pooled NgxConnections.
- Fixes a bug for requests that are finalized mid-IPRO lookup.
- Makes us use ngx_handle_read_event/ngx_del_event
In connection_read_handler(), make sure we act accordingly when
r->connection->error is set (indicating the the current request has
been finalized).
Reproduction of what happens when we don't: enable IPRO+SPDY, and
rapidly refresh a page with chrome. These rapid abortions will
eventually trigger a segfault/hang/misc bad behaviour.
For FetchInPlaceResource, NgxBaseFetch would send two bytes down its
pipe, one upon HeaderComplete() and one upon HandleDone(). We need
only one to resume processing on the nginx side.
There is a race between ps_connection_read_handler() and processing
of the byte send by NgxBaseFetch::HandleDone().
ps_connection_read_handler() clears the pipe when the request is
finalized, and also drains it on each event - so two writes could be
processed as one when lucky, masking the problem).
One concrete problem this solved for me was that SPDY + IPRO +
proxy_pass would segfault, hang, and/or pass on 5xx/404 responses
from IPRO lookup fetches to the browser, next to alerts about
r->count being zero in nginx's error.log
Might fix https://github.com/pagespeed/ngx_pagespeed/issues/788
Fixes https://github.com/pagespeed/ngx_pagespeed/issues/792
Based on @dinic his work, add keep-alive support for the native fetcher.
Adds a new option, usable at the http{} level in configuration:
pagespeed NativeFetcherMaxKeepaliveRequests 50;
The default value is 100 (aligned to nginx). Setting the value to 1 turns off
keep-alive requests altogether).
Most notable changes:
- Request keep-alive by adding the appropriate request header
- Fixes connections getting reused while they are servicing other requests:
- Remove connection from the pool of available connections for keepalive when applicable
- Disable keepalive in more appropriate situations
- Response parsing fixes
- Remove connections that timeout from the k.a. pool
- Add a few sanity (D)CHECKS
- Emit debug messages for traceability
- Fix for ignoring ipv6 addresses returned from dns queries when ipv6 is enabled.
- Bump the fetch timeout in test configuration to deflake tests that require dns
lookups (which will be done via 8.8.8.8 currently for the native fetcher)
Otto van der Schaaf