1137 Commits

Author SHA1 Message Date
Jeff Kaufman 14c7fb1c8d system test: use PAGESPEED_EXPECTED_FAILURES for known bad tests 2012-11-30 17:12:13 -05:00
Jeff Kaufman 8509618cb5 ngx_base_fetch: fix extra chunks and race condition
We were sometimes failing to set last_buf when we should, but being protected
from the consequences by another bug that made us usually send additional extra
buffers with last_buf set.

See discussion in #78, pull request was #94.
2012-11-29 09:39:33 -05:00
Jeff Kaufman 4b59d8badf caching: support memcached
Merge in Otto's #65 porting the apache memcached implementation to nginx.
2012-11-28 15:44:05 -05:00
Jeff Kaufman f0f952d7a6 housekeeping: use prefix ps_ instead of ngx_http_pagespeed_
Within the anonymous namespace in ngx_pagespeed.css we don't really need to be
using the long `ngx_http_pagespeed_` prefix.  Switching to just `ps_` should be
safe.  For symbols that need to be visible outside this file, though, we need to
keep the long prefix.

Fixes #85.
2012-11-28 11:23:17 -05:00
Jeff Kaufman 9e71fb76e2 Merge pull request #88 from pagespeed/jefftk-fix-indent
housekeeping: fix indentation
2012-11-27 15:04:53 -08:00
Jeff Kaufman 23c967f864 readme: project is somewhat more ready than it was 2012-11-27 16:26:31 -05:00
Jeff Kaufman 6491615bc8 configuration: handle multi-layer configuration
Nginx allows configuration at three levels:
 - main
 - server
 - location

Previously we did everything at the server level, which made for a few problems:
 - we didn't support options in location blocks
 - we created an NgxRewriteDriverFactory for every server block, which when
   combined with the Serf fetcher meant we had separate fetch pools for every
   server.  If someone had nginx configured with a lot of server blocks, we
   could attempt far too many simultaneous loopback fetches and DOS ourself.

Now we store things at several levels:
 - main
   - rewrite driver factory
 - server
   - most things
 - location
   - directory specific options
2012-11-27 15:07:04 -05:00
Jeff Kaufman 25901bacd1 housekeeping: fix indentation
Fix indentation on `ngx_http_pagespeed_writer` and
`ngx_http_pagespeed_write_helper`.
2012-11-27 11:21:41 -05:00
Jeff Kaufman 8ec05b79eb ngx_pagespeed.cc: fix double request finalization (issue 79)
When returning NGX_ERROR in the header filter, apperently nginx already
finalizes the http request for us. This fixes #79

It also puts a check in place for a switch label that should be impossible to
reach.

(Merged from oschaaf-issue-79.)
2012-11-27 10:43:10 -05:00
Jeff Kaufman e4f88950f6 Merge branch 'master' of github.com:pagespeed/ngx_pagespeed 2012-11-26 17:41:04 -05:00
Jeff Kaufman 263a6e4feb system test: set a maximum html size to process 2012-11-26 17:40:39 -05:00
Jeff Kaufman d41c90bad7 Merge pull request #31 from pagespeed/bnoordhuis-spinlocks
ngx_base_fetch: implement locking
2012-11-26 14:22:19 -08:00
Jeff Kaufman afb7105351 system test: add caching header 2012-11-26 17:18:17 -05:00
Jeff Kaufman 14342b5452 Merge pull request #77 from pagespeed/jefftk-merge-bug
options: fix merging
2012-11-21 12:57:13 -08:00
Jeff Kaufman daaedd96bb options: fix merging
The merging function was broken, in that it did

    child.Merge(parent)

when the semantics we actually need are

    child = parent.Clone().Merge(child)
2012-11-21 14:19:35 -05:00
Jeff Kaufman d7e07a8ff1 Merge pull request #76 from pagespeed/bnoordhuis-build-debug
build: link against debug libs if NGX_DEBUG=YES
2012-11-21 11:13:30 -08:00
Ben Noordhuis 4aad27ccf9 ngx_base_fetch: implement locking 2012-11-21 20:09:03 +01:00
Ben Noordhuis 51de81b123 build: link against debug libs if NGX_DEBUG=YES
Link against the debug versions of the mod_pagespeed libraries when nginx is
configured with --with-debug.

Requires that said libraries have been pre-built with BUILDTYPE=Debug.
2012-11-21 19:37:04 +01:00
Jeff Kaufman 4f3bf2d495 Merge pull request #71 from yaoweibin/fixed_pagespeed_write_fitler_wrong_output_filter
fixed the wrong next output filter in the pagespeed writer
2012-11-21 06:25:41 -08:00
Jeff Kaufman 06980882f9 Merge pull request #68 from pagespeed/jefftk-404-on-bad-static-urls
static javascript manager: 404 on not found
2012-11-21 06:25:13 -08:00
Weibin Yao 5702b8dd10 fixed the wrong next output filter in the pagespeed writer
This bug could cause only output part of the content with large file.
2012-11-21 17:54:37 +08:00
Jeff Kaufman 3ef24295ef static javascript manager: 404 on not found
If we get an invalid filename in a request for static pagespeed content, we
should return NGX_DECLINED from the handler so that nginx can 404 for us.

Fixes issue #67.
2012-11-20 11:06:51 -05:00
Jeff Kaufman 6071062407 Merge pull request #64 from pagespeed/jefftk-deferjs-system-test
content handler: support static javascript requests
2012-11-20 08:00:46 -08:00
Jeff Kaufman 0680b81028 Merge pull request #66 from pagespeed/oschaaf-fix-apr-includes
apr: include headers in a portable manner
2012-11-20 06:51:20 -08:00
Otto van der Schaaf d12508b12d apr: include headers in a portable manner 2012-11-20 15:21:56 +01:00
Jeff Kaufman 72b1b98bb9 content handler: support static javascript requests
Requests to:

  /ngx_pagespeed_static/...

can be handled synchronously by pagespeed.  They're used for support files for
optimizations, like js_defer.js.

This also includes a couple refactors:

 - I pulled most of NgxBaseFetch::CopyBufferToNginx() out into
   ngx_http_pagespeed_string_piece_to_buffer_chain().

 - ngx_http_pagespeed_create_request_context() now returns a
   CreateRequestContext::Response which gives callers more detailed information
   about the request.

 - I also pulled ngx_http_pagespeed_set_cache_control() out of
   ngx_http_pagespeed_header_filter().

Fixes issue #62.
2012-11-19 16:29:25 -05:00
Jeff Kaufman 318e14fa32 system test: label pre-compressed test data as such
The system test 'gzipped js origin' needs files in the directory
  /mod_pagespeed_test/compressed/
to be served as pre-gzip encoded javascript.
2012-11-19 11:30:09 -05:00
Jeff Kaufman eaef938988 base fetch: call ComputeCaching() on the pagespeed response headers
Merge of Otto's pull request #61
2012-11-19 11:07:22 -05:00
Jeff Kaufman e6bd7b7213 base fetch: populate the response status code
Copy the response status code from nginx to pagespeed.

Merge of Otto's pull request #60
2012-11-19 11:05:07 -05:00
Jeff Kaufman f22401af41 system test: /mod_pagespeed_test/no_cache/ needs to be cc:no-cache 2012-11-16 15:59:49 -05:00
Jeff Kaufman 2ab64cdff5 serf fetcher: minor cleanups 2012-11-15 13:26:40 -05:00
Jeff Kaufman 74831adad8 Merge pull request #54 from pagespeed/jefftk-fix-content-type
headers: content type
2012-11-14 15:03:42 -08:00
Jeff Kaufman 3c1e52c61c headers: content type
When setting content type you also need to set content_type_len and null out
content_type_lowcase or else downstream filters might look in the wrong place
for it.

See:
  http://stackoverflow.com/questions/8427528/nginx-gzip
  http://lxr.evanmiller.org/http/source/http/ngx_http_core_module.c#L1659
2012-11-14 16:03:02 -05:00
Jeff Kaufman a85de8cb68 move ngx_http_string_piece_to_pool_string out of anonymous namespace 2012-11-14 15:19:09 -05:00
Jeff Kaufman 16a9ed56bc Merge pull request #51 from pagespeed/jefftk-anonymous-namespace
ngx_pagespeed.cc: use an anonymous namespace
2012-11-14 12:17:05 -08:00
Jeff Kaufman 459eefd872 configuration: implement configuration parsing
Where mod pagespeed has a configuration option `ModPagespeedFooBar` we now
support `pagespeed FooBar` and `pagespeed ModPagespeedFooBar`.
2012-11-14 15:13:01 -05:00
Jeff Kaufman 76ebd7f96a use an anonymnous namespace instead of the static keyword to restrict visibility to this file 2012-11-13 11:43:57 -05:00
Jeff Kaufman bcda04288e Merge pull request #45 from yaoweibin/fixed_connection
Release the connection attached to the pipe
2012-11-13 08:05:42 -08:00
Jeff Kaufman aeb0de0e77 configuration: prepare to parse options
Set up the background code for connecting pagespeed configuration structures
(RewriteOptions) to the companion nginx ones.  Passes all the actual work
through to NgxRewriteOptions::ParseAndSetOptions which isn't written yet.
(All it does so far is interpret "pagespeed on/off".)

Adding nginx subclasses of RewriteOptions and ServerContext are a bit overkill
for this change, but we're going to need them a lot.

Once configuration is complete, any mod_pagespeed configuration directive should
work in ngx_pagespeed:

    mod_pagespeed.conf:
      ModPagespeedEnableFilters collapse_whitespace
      ModPagespeedRunExperiment on
      ModPagespeedExperimentSpec id=3;percent=50;default
      ModPagespeedExperimentSpec id=4;percent=50

    ngx_pagespeed.conf:
      pagespeed ModPagespeedEnableFilters collapse_whitespace
      pagespeed ModPagespeedRunExperiment on;
      pagespeed ModPagespeedExperimentSpec "id=3;percent=50;default";
      pagespeed ModPagespeedExperimentSpec "id=4;percent=50";

The `ModPagespeed` is optional, and discouraged:

    ngx_pagespeed.conf:
      pagespeed EnableFilters collapse_whitespace
      pagespeed RunExperiment on;
      pagespeed ExperimentSpec "id=3;percent=50;default";
      pagespeed ExperimentSpec "id=4;percent=50";
2012-11-13 10:26:41 -05:00
Jeff Kaufman 57a0f1f6ee Revert "Fetcher: switch us back to the wget fetcher"
This reverts commit 8a8066ec02.

Couldn't reproduce the problems with the serf fetcher, so switching back to it.
2012-11-13 09:33:59 -05:00
Jeff Kaufman 5e0e9df211 Caching: File System Cache
Merging in Otto's file system cache code from pull request #46.
2012-11-12 16:57:03 -05:00
Jeff Kaufman 8a8066ec02 Fetcher: switch us back to the wget fetcher
Commit b1eb547275, which added the serf fetcher, broke fetching.  For now,
switch back to the wget fetcher.
2012-11-12 16:52:58 -05:00
Otto van der Schaaf 35869d7679 Merge pull request #32 from pagespeed/oschaaf-apr-serf-test
this branch links in apr, aprutil and serf, and adds the serf fetcher
2012-11-09 13:35:29 -08:00
Otto van der Schaaf b1eb547275 apr and serf: link in apr, apr-utils, and serf
- Links in apr, apr-utils, and serf.
- Replaces the wget fetcher with serf, which will be replaced later
on with the native fetcher.
2012-11-09 22:29:51 +01:00
Weibin Yao 2c57b78517 Released the connection attached with the pipe
This problem could cause the connection and event leak with Nginx.
2012-11-09 16:25:45 +08:00
Jeff Kaufman 5d488f3281 fix obselete comment 2012-11-08 17:24:23 -05:00
Jeff Kaufman db53d1f7c2 headers: use headers generated by pagespeed
Pagespeed puts headers in ngx_base_fetch->response_headers() and nginx calls
NgxBaseFetch::CollectHeaders to get them out, which wasn't implemented before.

Other minor changes in this commit:
 - whitespace fixes
 - ordering of includes
 - ngx_http_pagespeed_str_to_string_piece now takes its argument by value
2012-11-08 13:03:38 -05:00
Jeff Kaufman 39224fb283 options: don't run furious when we have custom options
If custom_options were set by GetQueryOptions(), don't run the experiment
framework. We don't want experiments to be contaminated with unexpected
settings.
2012-11-08 10:39:58 -05:00
Jeff Kaufman 7b8854db1a ngx_pagespeed: respect ModPagespeed=off
One of the things you can do with headers or query params is turn off mod
pagespeed:

    curl -D- 'http://localhost:8050/mod_pagespeed_example/combine_css.html'
    curl -D- 'http://localhost:8050/mod_pagespeed_example/combine_css.html\
      ?ModPagespeed=off'

We weren't respecting this fully before, still removing the etag, dropping the
content length, etc.

merge of pull request #28
2012-11-08 10:33:39 -05:00
Jeff Kaufman a5d6c7046e enable core filters by default instead of just cache extension 2012-11-08 10:32:43 -05:00