Commit Graph

900 Commits

Author SHA1 Message Date
Otto van der Schaaf 22749d7222 Merge pull request #906 from pagespeed/oschaaf-trunk-tracking-module-position
module-position: optional switch to install at HTTP_AUX_FILTER_MODULES
2015-02-12 17:09:23 +01:00
Otto van der Schaaf a8141eadd3 module-position: optional switch to install at HTTP_AUX_FILTER_MODULES
This change adds a flag to optionally position the module at HTTP_AUX_FILTER_MODULES
instead of just in front of the gzip module. Doing so allows header module(s) to
fully manipulate any response headers before they are send to the user-agent.
With https://github.com/pagespeed/ngx_pagespeed/pull/880, SSI no longer hangs,
and other modules using subrequests should not do so either. SSI subrequests are
not optimized when ngx_pagespeed runs at HTTP_AUX_FILTER_MODULES, so the test for
it is added to the expected failures when the tests are run for this module position.

To build with the module respositioned:
  POSITION_AUX=true ./configure ......
The same variable can be set before running the system tests, in which case it will
perform additional header tests and add the SSI test to the expected failures.
2015-02-12 16:23:11 +01:00
Otto van der Schaaf 2d28ff38e8 Merge pull request #899 from pagespeed/oschaaf-trunk-tracking-shutdown
trunk-tracking-shutdown: improve shutdown code
2015-02-10 23:52:33 +01:00
Otto van der Schaaf d7f1c0dc48 Shutdown: improve shutdown handling
- Fix valgrind errors
- Add a test to make sure all logged output looks sane by whitelisting
  current errors/warnings.
- Stop our nginx test instances after we are done testing.
- Add tests for shutting down and reloading configuration under high
  load (depends on ab).
- Reduce the number of keepalive requests in the keepalive tests to speed
  up test runs.
- Fix exiting with open file descriptors, fix cleanup in nginx's cache
  manager/loader processes
- Attempt to finish up queued up NgxBaseFetches/requests on shutdown/reload
- Under valgrind the blocking rewrite started failing after adding a test
  for reloading configuration under high load.
  I've added it to the expected failures for valgrind, looking into this
  is up next.
- Decline in ps_resource_handler when nginx is quitting. This makes us
  more reliable on continued stress during shutdown/reload.
2015-02-10 23:13:44 +01:00
Jeffrey Crowell 737e9396f5 Merge pull request #892 from pagespeed/crowell-fix-888
Replace CHECK with return failure and log.
2015-01-30 10:28:04 -05:00
Jeffrey Crowell 81029dafa6 Replace CHECK with return failure and log.
Fixes #888 along with
https://code.google.com/p/modpagespeed/source/detail?r=4533
2015-01-30 10:23:00 -05:00
Otto van der Schaaf 001cd2dc4c Merge pull request #889 from pagespeed/oschaaf-trunk-tracking-content-handler-error-responses
content-handling-errors: fall back to nginx's default handling
2015-01-27 16:50:04 +01:00
Otto van der Schaaf 3efebb7948 content-handling-errors: fall back to nginx's default handling
This change passes on non succesful status codes for pagespeed
resources and other places where we act as a content handler to
nginx. This has two benefits:
- Instead of a blank page, the user agent receives a formatted
  and (hopefully customized and informative) response.
- Header modules are able to operate on that response, which was
  requested in https://github.com/pagespeed/ngx_pagespeed/issues/612#issuecomment-58855816
2015-01-26 22:13:15 +01:00
Otto van der Schaaf 02b67b79ad Merge pull request #885 from pagespeed/oschaaf-trunk-tracking-release-early
Release base fetch early, eliminate write_pending and fetch_done
2015-01-26 16:46:30 +01:00
Otto van der Schaaf d0858790e4 Merge pull request #884 from pagespeed/oschaaf-trunk-tracking-base-fetch
base-fetch: prevent unused constructions and prop. cache lookups
2015-01-23 20:09:45 +01:00
Otto van der Schaaf 4d945b1c59 Release base fetch early, eliminate write_pending and fetch_done
Get rid of write_pending & fetch_done flags on the request context.
Assume that having ctx->base_fetch set means we have outstanding
work and thus we are not done yet, for simplicity.
This also means we'll release the base fetch earlier which seems
like a good idea to me.
2015-01-23 15:07:14 +01:00
Otto van der Schaaf 7d965aced7 Merge pull request #880 from pagespeed/oschaaf-trunk-tracking-run-posted-requests
posted-requests: run posted requests when appropriate
2015-01-21 19:47:13 +01:00
Otto van der Schaaf a5411a1c7c base-fetch: prevent unused constructions and prop. cache lookups
- Don't construct `NgxBaseFetch` instances and call `Done(false)` later when it turns out it wasn't needed.
- Track the purpose of the base fetch instead of just wether it is an IPRO lookup or not for more informational tracing
  as well as avoiding constructing base fetches when their purpose isn't know yet.
- Don't initiate a property cache lookup when we won't need it (`ProxyFetchFactory::InitiatePropertyCacheLookup`)
- Improve IPRO handling of `NgxBaseFetch` and make it clearer. Indicate we want to supress further events when the
  base fetch was used to lookup an IPRO url, and the result was not indicative of something we can respond with to the
  browser.
- Remove unused `RequestRouting` constants
2015-01-20 02:19:47 +01:00
Otto van der Schaaf 8a1260a1d1 posted-requests: run posted requests when appropriate
Request termination code relies on posted requests being run.
One of the things this fixes is that we won't hang when configured
to run before the SSI module.

Avoids trouble when changing the module order, which helps
https://github.com/pagespeed/ngx_pagespeed/issues/612
2015-01-19 10:28:00 +01:00
Otto van der Schaaf f2e7b15d9c Merge pull request #879 from pagespeed/oschaaf-trunk-tracking-server-header
server-header: improve handling of server header
2015-01-17 20:28:34 +01:00
Otto van der Schaaf 5bc61a7092 server-header: improve handling of server header
- Fixes an issue in the html flow that would prevent overriding the
value of the 'Server' response-header.
- Add tests that ensure we emit a single and correct server header
in all flows when not overriding it.
- Add tests that ensure overriding the 'Server' response header
works. The resource and IPRO flow are added to the expected failures
as those are not working yet (and will be adressed in a follow-up).

Fixed https://github.com/pagespeed/ngx_pagespeed/issues/864 (html flow)
2015-01-17 20:27:45 +01:00
Otto van der Schaaf 03819f4e4a Merge pull request #876 from pagespeed/oschaaf-single-fd-fixed
Oschaaf single fd fixed
2015-01-16 15:53:06 +01:00
Otto van der Schaaf 7ef61d2580 base-fetch-leak: Shut down NgxEventConnection before deleting it
Shutdown will drain the pipe and process the last events which
will give nginx the opportunity to run the cleanup handlers on the
associated requests.
2015-01-10 22:45:04 +01:00
Otto van der Schaaf a73c096950 Revert "Merge pull request #872 from pagespeed/jefftk-revert-834"
This reverts commit 7bd90f7b3a, reversing
changes made to 58228564dd.
2015-01-10 22:07:21 +01:00
Jeff Kaufman 7bd90f7b3a Merge pull request #872 from pagespeed/jefftk-revert-834
Revert "NgxConnection: single event mechanism
2015-01-09 10:25:12 -05:00
Jeff Kaufman 73981ffe1a Revert "NgxConnection: single event mechanism for NgxFetch and NgxBaseFetch"
This reverts commit 5dfe42f3d3.
2015-01-08 13:02:19 -05:00
Jeff Kaufman 58228564dd Merge pull request #869 from pagespeed/jefftk-system-moved
mod_pagespeed moved the location of the system directory; update our includes
2015-01-06 09:50:11 -05:00
Jeff Kaufman 340c86a49e mod_pagespeed moved the location of the system directory; update our includes 2015-01-06 09:22:56 -05:00
Otto van der Schaaf abdafc7aa4 Merge pull request #834 from pagespeed/oschaaf-single-pipe2
NgxConnection: single event mechanism for NgxFetch and NgxBaseFetch
2014-12-30 21:52:07 +01:00
Jan-Willem Maessen 8cf10873d1 Merge pull request #863 from pagespeed/jmaessen-ngx-message-handler-fix
Make ngx_message_handler work with new message_handler api.
2014-12-19 16:13:26 -05:00
Jan-Willem Maessen 2496716e36 Make ngx_message_handler work with new message_handler api. 2014-12-19 15:11:55 -05:00
Jeff Kaufman af364344e9 Merge pull request #862 from pagespeed/jefftk-rmdir
testing: give '-p' to mkdir
2014-12-17 12:55:44 -05:00
Jeff Kaufman 9da45910c7 rm won't remove the containing directory if a file is removed out from under it, which means the directory will still be there but will be empty. This is not a problem, so pass '-p' to not fail on already existing directories. 2014-12-17 12:50:23 -05:00
Otto van der Schaaf 5dfe42f3d3 NgxConnection: single event mechanism for NgxFetch and NgxBaseFetch
Abstract the pipe communication into NgxEventConnection, for reuse
by NgxBaseFetch and NgxUrlAsyncFetcher.

Based on Chai's earlier work, but with a few fixes discovered
while working on this and SPDY module compatibility

- Uses less file descriptors, I expect this to be faster but need
  measurement is needed to back that.
- Fixed NgxUrlAsyncFetcher actually shutting down its fetchers.
- Fixes a bug where we wouldn't clean idle pooled NgxConnections.
- Fixes a bug for requests that are finalized mid-IPRO lookup.
- Makes us use ngx_handle_read_event/ngx_del_event
2014-12-14 02:29:40 +01:00
Jeff Kaufman bff15040be Merge pull request #857 from pagespeed/jefftk-upstream-check-failure-function
Upstream the change to check_failures_and_exit, moving it to mod_pagespeed
2014-12-04 15:42:31 -05:00
Jeff Kaufman 3c5c5076de Upstream the change to check_failures_and_exit, moving it to mod_pagespeed 2014-12-04 13:22:13 -05:00
Jeff Kaufman eab4f8672f Merge pull request #852 from pagespeed/jefftk-move-tests-to-system
Move duplicate tests from the nginx-specific system test to system
2014-12-02 20:12:52 -05:00
Jeff Kaufman 852e376130 Move duplicate tests from the nginx-specific system test to system/system_test
As system/system_test.sh is in the mod_pagespeed repo, this is just a bunch of
deletes.
2014-12-02 15:11:23 -05:00
Jeff Kaufman b7529a8d9a Merge pull request #855 from pagespeed/jefftk-fix-tempdir
Rework parallization separation
2014-12-02 13:34:03 -05:00
Jeff Kaufman 5dd93fffae Rework parallization separation 2014-12-01 10:49:08 -05:00
Jeff Kaufman 48d23f10ba Merge pull request #851 from pagespeed/jefftk-fix-url-purging
Fix error with url cache purging test.  Followup to #850.
2014-11-17 11:25:03 -05:00
Jeff Kaufman f924bc72d9 Fix error with url cache purging test. Followup to #850. 2014-11-17 11:19:51 -05:00
Otto van der Schaaf 1aff187d0e Merge pull request #850 from pagespeed/oschaaf-trunk-tracking-script-filters-test
script-filters: Add a test for modifying filters via script variables
2014-11-14 16:03:22 +01:00
Jeff Kaufman 3b64df255a Merge pull request #849 from pagespeed/jefftk-prioritize-critical-css-test
testing: allow the prioritize_critical_css post test to fail
2014-11-14 10:02:49 -05:00
Otto van der Schaaf 0eeeca6e47 script-filters: Add a test for modifying filters via script variables
As requested via https://github.com/pagespeed/ngx_pagespeed/pull/822
2014-11-14 15:56:13 +01:00
Jeff Kaufman 6f68ba2f68 testing: allow the prioritize_critical_css post test to fail 2014-11-14 08:56:37 -05:00
Otto van der Schaaf e6e70c32a8 Merge pull request #822 from pagespeed/oschaaf-trunk-tracking-script-filters
script-filters: Allow using script variables in Enable/DisableFilters
2014-11-06 22:35:44 +01:00
Jeffrey Crowell a3e20a05e2 Merge pull request #846 from pagespeed/oschaaf-trunk-tracking-conf-multi-typo
NGX_CONF_MULTI: Typo
2014-11-04 13:46:55 -05:00
Otto van der Schaaf e28255e2ef NGX_CONF_MULTI: Typo
Fix accidental double pipe symbol.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/843
2014-11-04 13:29:49 +01:00
Otto van der Schaaf 10ac8ae4a3 Merge pull request #798 from pagespeed/oschaaf-trunk-tracking-conf-multi
older-nginx-versions: Add NGX_CONF_MULTI to ps_command flags.
2014-10-30 15:13:27 +01:00
Jeffrey Crowell 1b3ca08e22 Merge pull request #841 from pagespeed/crowell-tengine-stable-support
Check type of resolver_ctx->addrs.
2014-10-29 14:59:51 -04:00
Jeffrey Crowell de5bc5d943 Check type of resolver_ctx->addrs.
Check the type of resolver_ctx->addrs and make sure that it is
ngx_addr_t* instead of in_addr_t*. addresses issue #839
2014-10-29 14:49:55 -04:00
Otto van der Schaaf afd0f01686 script-filters: Allow using script variables in Enable/DisableFilters
To support a non-evil way (if-in-location) of tuning SPDY specific
configuration, allow scripting of Enable/DisableFilters in nginx.conf
2014-10-10 17:33:44 +02:00
Otto van der Schaaf 0cdb81fab4 Merge pull request #795 from pagespeed/oschaaf-native-fetcher-remove-check
native-fetcher: remove a DCHECK that possibly fires on shutdown.
2014-10-06 17:04:56 +02:00
hillsp b492d3966e Merge pull request #811 from pagespeed/cheesy-device-matching
Support minor PSOL API change for matches_device_type
2014-09-22 17:38:06 -04:00