Commit Graph

217 Commits

Author SHA1 Message Date
Joshua Marantz 07a6647b65 respect purge requests when serving ipro requests from ngx_pagespeed (#1193)
* respect purge requests when serving ipro requests from ngx_pagespeed

* Add TODO to share common base with ApacheFetch.

During system-tests, don't write into the source-controlled htdocs area. (#1197)

Instead, use a new (not-yet-committed) target in Makefile.tests to build
a mirror htdocs in test/tmp/root that sym-links the large readonly
section and deep copies the test areas for purge and flush tests.

Pass the install-path into Makefile (#1200)

* Pass the install-path into Makefile, rather than having the Makefile
depend on PWD.

* Move the double-quotes be before the value, to be consistent with other
Makefiles.
2016-07-25 12:58:04 -04:00
Jeff Kaufman 84ab39e8e3 Disable the multiple vary headers test for 1.11 2016-03-30 15:27:36 -04:00
Otto van der Schaaf 2b4c097d48 vary-header: Emit a single vary header in the IPRO flow
The report from some time ago mentioned three Vary: headers,
but I can now only reproduce two using trunk-tracking plus the
original repro-configuration.

This fix unflags r->gzip_vary as set by the gzip module when PSOL
hands us Vary: Accept-Encoding, to make sure that nginx's core
header filter doesn't append another one.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/1064
2016-03-30 14:59:07 -04:00
Jeff Kaufman 371293af48 handle additional warnings 2016-03-28 09:04:29 -04:00
Otto van der Schaaf 88e3d48d23 Fix crasher on 404 .pagespeed. resources w/a custom location
- Fix nginx-side flow so we handle .pagespeed. resources ok
  when they will land on a customized 404 internal location.
- Additionally, check for a wiped request context and make sure
  we do not dereference a null pointer, which is what hurt in
  the flow we entered above as the IPRO lookup still was
  generating events while the nginx side request context was
  gone.
- Also, as a preliminary measure, do not check fail when we
  receive a stale event originating from a NgxBaseFetch that
  is no longer associated with our request context.
  Do log a warning so we'll hear about this happening either
  through system test failures or a bug report.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/1081
2016-03-03 14:57:40 -05:00
Joshua Marantz 14aa4fd962 suppress errors on new 2-pass and 3-pass ipro tests 2015-12-02 15:01:37 -05:00
Joshua Marantz 706577a34b do not consider it a test failure to get a slow ReadFile operation. 2015-12-02 08:27:47 -05:00
Otto van der Schaaf 40c05b4a4f location-header: Be careful with headers_out->location
Only set headers_out->location when the upstream originally did
as well. If the Location: header value involved starts with "/"
nginx will absolutify it, ignoring any X-Forwarded-Proto header
in the process.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/819
(Confirmed: https://github.com/pagespeed/ngx_pagespeed/issues/1029)
Hopefully fixes https://github.com/pagespeed/ngx_pagespeed/issues/711
2015-12-01 18:55:02 +01:00
Jeff Kaufman 4628e1c0ea Merge pull request #1035 from pagespeed/jefftk-script-var-testing
Support script variables in ShardDomain
2015-11-19 11:01:41 -08:00
Jeff Kaufman ea82f0de19 Support script variables in ShardDomain 2015-11-02 10:08:00 -05:00
Jeff Kaufman 5f312820e5 Merge pull request #974 from pagespeed/jefftk-handler-restrictions
Add support for blocking access to our handlers.
2015-11-02 08:18:10 -05:00
Otto van der Schaaf 634b813071 IPRO+MPD: Make In-Place-Resource-Optimization with MapProxyDomain work
Fixes https://github.com/pagespeed/ngx_pagespeed/issues/1015

Pull with MPS Side for this change:
https://github.com/pagespeed/mod_pagespeed/pull/1161
2015-10-27 18:11:35 +01:00
Jeff Kaufman fe47eeb4f3 Add support for blocking access to our handlers.
Nginx side of the fix for https://github.com/pagespeed/mod_pagespeed/issues/1088
2015-09-18 09:03:01 -04:00
Joshua Marantz 8bac32c669 ignore errors about slow file operations, which can occur when running valgrind tests 2015-07-31 16:57:20 -04:00
Jeffrey Crowell 9ee745c2ef Revert "temporarily comment tests using modpagespeed.com"
This reverts commit 5337e4b558.

closes #984
2015-07-07 13:15:33 -04:00
Jeffrey Crowell 5337e4b558 temporarily comment tests using modpagespeed.com
revert this commit when modpagespeed.com is back up
2015-07-06 12:36:44 -04:00
Joshua Marantz 8464a00a77 nginx system test changes to repro issue https://github.com/pagespeed/mod_pagespeed/issues/1077 2015-05-18 09:46:12 -04:00
Jeff Kaufman 137792e88f Change references from net/instaweb/automatic to pagespeed/automatic. 2015-03-13 09:30:14 -04:00
Jeffrey Crowell 6e1a9a73a5 rm "unexpected response" grep from system test 2015-03-05 14:02:55 -05:00
Jeffrey Crowell 88d9b5241d Merge remote-tracking branch 'origin/trunk-tracking' into crowell_rconfig 2015-02-25 16:47:48 -05:00
Otto van der Schaaf effe857083 native-fetcher: Fix a few bugs
- Add handling for header only responses
- Fixup nginx i/o handling
- Fix issues in parser state
- Revert timeout for native fetcher dns resolve to 10 seconds
- Ignore warning that gets logged with this change in tests during
  fetching from a host that refuses connections.

Should help stabilizing (valgrind) test runs some more.
2015-02-25 21:03:01 +01:00
Jeffrey Crowell c26313e7ab Port remote configuration option to nginx. 2015-02-23 11:25:17 -05:00
Otto van der Schaaf 58b1264745 native-fetcher: bump dns timeout to deflake running the tests
Nginx sometimes receives an expected response from 8.8.8.8 when
resolving host names, and the subsequent retry doesn't finish up
in 10 seconds. Bumping the timeout helps, but does not fully
explain why nginx is sometimes receiving an unexpected response.
I'll create a separate issue for investigating that.
2015-02-17 11:05:34 +01:00
Otto van der Schaaf 62c8947b2d configuration-reload-test: wait for the new worker to get ready
After sending a SIGHUP, wait for the new worker process with the
new configuration to handle further requests, or else the tests
running after that might get confused when the new worker becomes
active with fresh shared mem statistics and shared mem cache.
2015-02-12 18:08:18 +01:00
Otto van der Schaaf 22749d7222 Merge pull request #906 from pagespeed/oschaaf-trunk-tracking-module-position
module-position: optional switch to install at HTTP_AUX_FILTER_MODULES
2015-02-12 17:09:23 +01:00
Otto van der Schaaf a8141eadd3 module-position: optional switch to install at HTTP_AUX_FILTER_MODULES
This change adds a flag to optionally position the module at HTTP_AUX_FILTER_MODULES
instead of just in front of the gzip module. Doing so allows header module(s) to
fully manipulate any response headers before they are send to the user-agent.
With https://github.com/pagespeed/ngx_pagespeed/pull/880, SSI no longer hangs,
and other modules using subrequests should not do so either. SSI subrequests are
not optimized when ngx_pagespeed runs at HTTP_AUX_FILTER_MODULES, so the test for
it is added to the expected failures when the tests are run for this module position.

To build with the module respositioned:
  POSITION_AUX=true ./configure ......
The same variable can be set before running the system tests, in which case it will
perform additional header tests and add the SSI test to the expected failures.
2015-02-12 16:23:11 +01:00
Otto van der Schaaf d7f1c0dc48 Shutdown: improve shutdown handling
- Fix valgrind errors
- Add a test to make sure all logged output looks sane by whitelisting
  current errors/warnings.
- Stop our nginx test instances after we are done testing.
- Add tests for shutting down and reloading configuration under high
  load (depends on ab).
- Reduce the number of keepalive requests in the keepalive tests to speed
  up test runs.
- Fix exiting with open file descriptors, fix cleanup in nginx's cache
  manager/loader processes
- Attempt to finish up queued up NgxBaseFetches/requests on shutdown/reload
- Under valgrind the blocking rewrite started failing after adding a test
  for reloading configuration under high load.
  I've added it to the expected failures for valgrind, looking into this
  is up next.
- Decline in ps_resource_handler when nginx is quitting. This makes us
  more reliable on continued stress during shutdown/reload.
2015-02-10 23:13:44 +01:00
Otto van der Schaaf 3efebb7948 content-handling-errors: fall back to nginx's default handling
This change passes on non succesful status codes for pagespeed
resources and other places where we act as a content handler to
nginx. This has two benefits:
- Instead of a blank page, the user agent receives a formatted
  and (hopefully customized and informative) response.
- Header modules are able to operate on that response, which was
  requested in https://github.com/pagespeed/ngx_pagespeed/issues/612#issuecomment-58855816
2015-01-26 22:13:15 +01:00
Otto van der Schaaf 5bc61a7092 server-header: improve handling of server header
- Fixes an issue in the html flow that would prevent overriding the
value of the 'Server' response-header.
- Add tests that ensure we emit a single and correct server header
in all flows when not overriding it.
- Add tests that ensure overriding the 'Server' response header
works. The resource and IPRO flow are added to the expected failures
as those are not working yet (and will be adressed in a follow-up).

Fixed https://github.com/pagespeed/ngx_pagespeed/issues/864 (html flow)
2015-01-17 20:27:45 +01:00
Jeff Kaufman 340c86a49e mod_pagespeed moved the location of the system directory; update our includes 2015-01-06 09:22:56 -05:00
Jeff Kaufman 9da45910c7 rm won't remove the containing directory if a file is removed out from under it, which means the directory will still be there but will be empty. This is not a problem, so pass '-p' to not fail on already existing directories. 2014-12-17 12:50:23 -05:00
Jeff Kaufman 3c5c5076de Upstream the change to check_failures_and_exit, moving it to mod_pagespeed 2014-12-04 13:22:13 -05:00
Jeff Kaufman 852e376130 Move duplicate tests from the nginx-specific system test to system/system_test
As system/system_test.sh is in the mod_pagespeed repo, this is just a bunch of
deletes.
2014-12-02 15:11:23 -05:00
Jeff Kaufman 5dd93fffae Rework parallization separation 2014-12-01 10:49:08 -05:00
Otto van der Schaaf 1aff187d0e Merge pull request #850 from pagespeed/oschaaf-trunk-tracking-script-filters-test
script-filters: Add a test for modifying filters via script variables
2014-11-14 16:03:22 +01:00
Otto van der Schaaf 0eeeca6e47 script-filters: Add a test for modifying filters via script variables
As requested via https://github.com/pagespeed/ngx_pagespeed/pull/822
2014-11-14 15:56:13 +01:00
Jeff Kaufman 6f68ba2f68 testing: allow the prioritize_critical_css post test to fail 2014-11-14 08:56:37 -05:00
Joshua Marantz 7ce289346a Override the method used to specify the format for conf files, so we can put an accurate error message in the admin GUI, and test it 2014-08-29 09:13:43 -04:00
Joshua Marantz a3f2e4766c use fgrep for counting ".pagespeed.", rather than grep with backslashed dot, which matches too much 2014-08-26 15:02:10 -04:00
Joshua Marantz a458a65d4e Update system test to be compatible with a helper-script update that makes start_test clear WGET_ARGS, making test stanzas more independent 2014-08-20 12:59:04 -04:00
Jeff Kaufman cad618dff7 testing: make it clearer why test flake in Issue #774 happens 2014-08-13 13:53:22 -04:00
Jeff Kaufman e6c5bc1b2e testing: direct port of r4146 to fix message color issue 2014-08-13 10:13:13 -04:00
Otto van der Schaaf d6d3b90ea3 Merge pull request #769 from pagespeed/oschaaf-large-beacon-posts
beacon response body processing from temp file
2014-08-12 17:57:34 +02:00
Otto van der Schaaf d7afd2acbc beacon response body processing from temp file
Add support for processing beacon posts with data sizes that exceed
the default or configured client_body_buffer_size, plus a system test
to ensure it.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/752
2014-08-12 07:46:30 +02:00
Maks Orlovich fe63bbf53f Port over test adjustment. 2014-08-11 16:26:20 -04:00
Otto van der Schaaf 8f6bb0c9e3 rewriting-posts: Support POST responses for html
Allow rewriting html responses to POST requests

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/757
2014-08-07 16:02:23 +02:00
jmarantz 2bfc05b4c6 Merge pull request #763 from pagespeed/jmarantz-http-200-ok-function
use a new helper method to verify HTTP response codes are 200 OK
2014-08-04 11:00:03 -04:00
Joshua Marantz c073a3120b remove debug comments 2014-08-04 10:02:31 -04:00
Joshua Marantz 5e3b2f23a5 Make script execute properly in the context of 'set -e' 2014-08-01 09:29:08 -04:00
Joshua Marantz f41ecf4010 use a new helper method to verify HTTP response codes are 200 OK 2014-08-01 09:19:40 -04:00