Commit Graph

234 Commits

Author SHA1 Message Date
Joshua Marantz 0baa230bd7 Add default setting for whether to test with a controller port. (#1278)
* Add default setting for whether to test with a controller port.

* Use RUN_CONTROLLER_TEST, which will actually be defaulted in
system/system_test.sh.  Comment out the controller port configuration
in the default nginx template.

* Finish up the conditionalization of the babysitter tests.

* Run the reload tests even with the controller off.

* clean up the diffs a little.
2016-09-20 15:42:47 -04:00
Joshua Marantz 1118910228 the remote config test is in a separate script now, so run it from the master test script (#1260) 2016-09-19 19:25:24 -04:00
hillsp c8f8aff23f Add GLOBAL_STATISTICS_URL variable (#1270) 2016-09-12 13:52:06 -04:00
Otto van der Schaaf 76eeec5972 special-responses: fix for hanging sometimes on special responses (#1238)
After calling ngx_http_filter_finalize_request return NGX_DONE
as we are still in pre-access phase. This bug shows up when the
"special" response is generated from, for example, php as opposed
to a static file.

Second part of the fix for
https://github.com/pagespeed/ngx_pagespeed/issues/1146
2016-09-07 10:49:53 +02:00
Joshua Marantz 9b47332d9c allow sloppy exit (#1258)
* set up a test subdir with cc:public

* allow 'adding function to sequence' warnings on shutdown, without breaking the nginx debug tests
2016-08-29 17:06:26 -04:00
Jeff Kaufman f8f32efb97 Test shm checkpointing. Relies on shm checkpointing implementation in mod_pagespeed. (#930) 2016-08-23 07:46:20 -04:00
Otto van der Schaaf ebe7c61f4f Make ngx_pagespeed behave like mod_pagespeed with regard to Flushing (#1217)
This change makes ngx_pagespeed listen to the FollowFlushes option.
When set to on (=default), ngx_pagespeed will forward incoming flushes
to ProxyFetch. When writing output, we'll now also set the flush flag on
the buffers we are about to send downstream.

Companion to mps commit: https://github.com/pagespeed/mod_pagespeed/commit/02de03e825bbd1f8d4ad4e1a1bef5263a16f3857
2016-07-12 12:26:38 -04:00
hillsp 1bb10ec599 Suppress controller warnings in test, fix up kill signals (#1210)
* Suppress controller warnings in test, fix up kill signals

* Switch valgrind shutdown to QUIT
2016-06-10 14:54:49 -04:00
Joshua Marantz d7f48d88c6 Pass the install-path into Makefile (#1200)
* Pass the install-path into Makefile, rather than having the Makefile
depend on PWD.

* Move the double-quotes be before the value, to be consistent with other
Makefiles.
2016-05-16 13:32:28 -04:00
Steve Hill fd26b873e9 Remove stray -q on pgrep. 2016-05-13 17:47:09 -04:00
hillsp d6c395151f Fix nginx_system_test so that it doesn't wait for verify_nginx_release.sh (#1199)
or wait indefinitely if it can't kill nginx.
2016-05-13 16:06:38 -04:00
Joshua Marantz dab6b8ac04 During system-tests, don't write into the source-controlled htdocs area. (#1197)
Instead, use a new (not-yet-committed) target in Makefile.tests to build
a mirror htdocs in test/tmp/root that sym-links the large readonly
section and deep copies the test areas for purge and flush tests.
2016-05-13 08:44:14 -04:00
Joshua Marantz be7837507f respect purge requests when serving ipro requests from ngx_pagespeed (#1193)
* respect purge requests when serving ipro requests from ngx_pagespeed

* Add TODO to share common base with ApacheFetch.
2016-05-11 11:08:04 -04:00
hillsp 090ec03a68 System test updates for CentralControllerPort. (#1182) 2016-04-28 15:45:29 -04:00
Jeff Kaufman 297b4ab34b handle additional warnings 2016-03-29 10:07:09 -04:00
Jeff Kaufman c3f2858e31 controller: don't fail tests when running in emacs
Apparently, when running in emacs and piped into another command, ps limits
it's output to COLUMNS making check_process_names fail.  Telling ps to use
wide, wide output with 'ww' fixes this.

Additionally, use check_from so that if this test does fail we can debug
it better, seeing what grep was looking at.
2016-03-15 16:36:52 -04:00
Jeff Kaufman a9142f0003 Merge pull request #1105 from pagespeed/oschaaf-trunk-tracking-issue-1064
vary-header: Emit a single vary header in the IPRO flow
2016-03-08 06:21:46 -08:00
Jeff Kaufman 8efb831ba4 Support centralized process changes in github.com/pagespeed/mod_pagespeed/commit/9bf7523 2016-03-07 16:14:43 -05:00
Otto van der Schaaf b081bb7aec vary-header: Emit a single vary header in the IPRO flow
The report from some time ago mentioned three Vary: headers,
but I can now only reproduce two using trunk-tracking plus the
original repro-configuration.

This fix unflags r->gzip_vary as set by the gzip module when PSOL
hands us Vary: Accept-Encoding, to make sure that nginx's core
header filter doesn't append another one.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/1064
2016-01-28 03:37:01 +01:00
Joshua Marantz 9e6048f771 Merge pull request #1092 from pagespeed/jmarantz-ipro-flow-cache-not-flaky
remove 'IPRO flow uses cache as expected' from flaky-list, ...
2016-01-05 15:11:46 -05:00
Joshua Marantz a512d95258 remove 'IPRO flow uses cache as expected' from flaky-list, but strip out 'Fetch timed out' from log-warning check 2016-01-05 11:46:35 -05:00
Otto van der Schaaf 1964ef5219 Fix crasher on 404 .pagespeed. resources w/a custom location
- Fix nginx-side flow so we handle .pagespeed. resources ok
  when they will land on a customized 404 internal location.
- Additionally, check for a wiped request context and make sure
  we do not dereference a null pointer, which is what hurt in
  the flow we entered above as the IPRO lookup still was
  generating events while the nginx side request context was
  gone.
- Also, as a preliminary measure, do not check fail when we
  receive a stale event originating from a NgxBaseFetch that
  is no longer associated with our request context.
  Do log a warning so we'll hear about this happening either
  through system test failures or a bug report.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/1081
2015-12-28 15:59:13 +01:00
Joshua Marantz 14aa4fd962 suppress errors on new 2-pass and 3-pass ipro tests 2015-12-02 15:01:37 -05:00
Joshua Marantz 706577a34b do not consider it a test failure to get a slow ReadFile operation. 2015-12-02 08:27:47 -05:00
Otto van der Schaaf 40c05b4a4f location-header: Be careful with headers_out->location
Only set headers_out->location when the upstream originally did
as well. If the Location: header value involved starts with "/"
nginx will absolutify it, ignoring any X-Forwarded-Proto header
in the process.

Fixes https://github.com/pagespeed/ngx_pagespeed/issues/819
(Confirmed: https://github.com/pagespeed/ngx_pagespeed/issues/1029)
Hopefully fixes https://github.com/pagespeed/ngx_pagespeed/issues/711
2015-12-01 18:55:02 +01:00
Jeff Kaufman 4628e1c0ea Merge pull request #1035 from pagespeed/jefftk-script-var-testing
Support script variables in ShardDomain
2015-11-19 11:01:41 -08:00
Jeff Kaufman ea82f0de19 Support script variables in ShardDomain 2015-11-02 10:08:00 -05:00
Jeff Kaufman 5f312820e5 Merge pull request #974 from pagespeed/jefftk-handler-restrictions
Add support for blocking access to our handlers.
2015-11-02 08:18:10 -05:00
Otto van der Schaaf 634b813071 IPRO+MPD: Make In-Place-Resource-Optimization with MapProxyDomain work
Fixes https://github.com/pagespeed/ngx_pagespeed/issues/1015

Pull with MPS Side for this change:
https://github.com/pagespeed/mod_pagespeed/pull/1161
2015-10-27 18:11:35 +01:00
Jeff Kaufman fe47eeb4f3 Add support for blocking access to our handlers.
Nginx side of the fix for https://github.com/pagespeed/mod_pagespeed/issues/1088
2015-09-18 09:03:01 -04:00
Joshua Marantz 8bac32c669 ignore errors about slow file operations, which can occur when running valgrind tests 2015-07-31 16:57:20 -04:00
Jeffrey Crowell 9ee745c2ef Revert "temporarily comment tests using modpagespeed.com"
This reverts commit 5337e4b558.

closes #984
2015-07-07 13:15:33 -04:00
Jeffrey Crowell 5337e4b558 temporarily comment tests using modpagespeed.com
revert this commit when modpagespeed.com is back up
2015-07-06 12:36:44 -04:00
Joshua Marantz 8464a00a77 nginx system test changes to repro issue https://github.com/pagespeed/mod_pagespeed/issues/1077 2015-05-18 09:46:12 -04:00
Jeff Kaufman 137792e88f Change references from net/instaweb/automatic to pagespeed/automatic. 2015-03-13 09:30:14 -04:00
Jeffrey Crowell 6e1a9a73a5 rm "unexpected response" grep from system test 2015-03-05 14:02:55 -05:00
Jeffrey Crowell 88d9b5241d Merge remote-tracking branch 'origin/trunk-tracking' into crowell_rconfig 2015-02-25 16:47:48 -05:00
Otto van der Schaaf effe857083 native-fetcher: Fix a few bugs
- Add handling for header only responses
- Fixup nginx i/o handling
- Fix issues in parser state
- Revert timeout for native fetcher dns resolve to 10 seconds
- Ignore warning that gets logged with this change in tests during
  fetching from a host that refuses connections.

Should help stabilizing (valgrind) test runs some more.
2015-02-25 21:03:01 +01:00
Jeffrey Crowell c26313e7ab Port remote configuration option to nginx. 2015-02-23 11:25:17 -05:00
Otto van der Schaaf 58b1264745 native-fetcher: bump dns timeout to deflake running the tests
Nginx sometimes receives an expected response from 8.8.8.8 when
resolving host names, and the subsequent retry doesn't finish up
in 10 seconds. Bumping the timeout helps, but does not fully
explain why nginx is sometimes receiving an unexpected response.
I'll create a separate issue for investigating that.
2015-02-17 11:05:34 +01:00
Otto van der Schaaf 62c8947b2d configuration-reload-test: wait for the new worker to get ready
After sending a SIGHUP, wait for the new worker process with the
new configuration to handle further requests, or else the tests
running after that might get confused when the new worker becomes
active with fresh shared mem statistics and shared mem cache.
2015-02-12 18:08:18 +01:00
Otto van der Schaaf 22749d7222 Merge pull request #906 from pagespeed/oschaaf-trunk-tracking-module-position
module-position: optional switch to install at HTTP_AUX_FILTER_MODULES
2015-02-12 17:09:23 +01:00
Otto van der Schaaf a8141eadd3 module-position: optional switch to install at HTTP_AUX_FILTER_MODULES
This change adds a flag to optionally position the module at HTTP_AUX_FILTER_MODULES
instead of just in front of the gzip module. Doing so allows header module(s) to
fully manipulate any response headers before they are send to the user-agent.
With https://github.com/pagespeed/ngx_pagespeed/pull/880, SSI no longer hangs,
and other modules using subrequests should not do so either. SSI subrequests are
not optimized when ngx_pagespeed runs at HTTP_AUX_FILTER_MODULES, so the test for
it is added to the expected failures when the tests are run for this module position.

To build with the module respositioned:
  POSITION_AUX=true ./configure ......
The same variable can be set before running the system tests, in which case it will
perform additional header tests and add the SSI test to the expected failures.
2015-02-12 16:23:11 +01:00
Otto van der Schaaf d7f1c0dc48 Shutdown: improve shutdown handling
- Fix valgrind errors
- Add a test to make sure all logged output looks sane by whitelisting
  current errors/warnings.
- Stop our nginx test instances after we are done testing.
- Add tests for shutting down and reloading configuration under high
  load (depends on ab).
- Reduce the number of keepalive requests in the keepalive tests to speed
  up test runs.
- Fix exiting with open file descriptors, fix cleanup in nginx's cache
  manager/loader processes
- Attempt to finish up queued up NgxBaseFetches/requests on shutdown/reload
- Under valgrind the blocking rewrite started failing after adding a test
  for reloading configuration under high load.
  I've added it to the expected failures for valgrind, looking into this
  is up next.
- Decline in ps_resource_handler when nginx is quitting. This makes us
  more reliable on continued stress during shutdown/reload.
2015-02-10 23:13:44 +01:00
Otto van der Schaaf 3efebb7948 content-handling-errors: fall back to nginx's default handling
This change passes on non succesful status codes for pagespeed
resources and other places where we act as a content handler to
nginx. This has two benefits:
- Instead of a blank page, the user agent receives a formatted
  and (hopefully customized and informative) response.
- Header modules are able to operate on that response, which was
  requested in https://github.com/pagespeed/ngx_pagespeed/issues/612#issuecomment-58855816
2015-01-26 22:13:15 +01:00
Otto van der Schaaf 5bc61a7092 server-header: improve handling of server header
- Fixes an issue in the html flow that would prevent overriding the
value of the 'Server' response-header.
- Add tests that ensure we emit a single and correct server header
in all flows when not overriding it.
- Add tests that ensure overriding the 'Server' response header
works. The resource and IPRO flow are added to the expected failures
as those are not working yet (and will be adressed in a follow-up).

Fixed https://github.com/pagespeed/ngx_pagespeed/issues/864 (html flow)
2015-01-17 20:27:45 +01:00
Jeff Kaufman 340c86a49e mod_pagespeed moved the location of the system directory; update our includes 2015-01-06 09:22:56 -05:00
Jeff Kaufman 9da45910c7 rm won't remove the containing directory if a file is removed out from under it, which means the directory will still be there but will be empty. This is not a problem, so pass '-p' to not fail on already existing directories. 2014-12-17 12:50:23 -05:00
Jeff Kaufman 3c5c5076de Upstream the change to check_failures_and_exit, moving it to mod_pagespeed 2014-12-04 13:22:13 -05:00
Jeff Kaufman 852e376130 Move duplicate tests from the nginx-specific system test to system/system_test
As system/system_test.sh is in the mod_pagespeed repo, this is just a bunch of
deletes.
2014-12-02 15:11:23 -05:00