add not_component argument for ngx.req.escape_uri

.mergify.yml

@@ -0,0 +1,35 @@
+---
+pull_request_rules:
+  - name: warn on conflicts
+    conditions:
+      - conflict
+    actions:
+      comment:
+        message: This pull request is now in conflict :(
+      label:
+        add:
+          - conflict
+  - name: remove conflict label if not needed
+    conditions:
+      - -conflict
+    actions:
+      label:
+        remove:
+          - conflict
+  - name: add label needs-test-cases
+    conditions:
+      - files~=^src/
+      - -files~=^t/
+    actions:
+      label:
+        add:
+          - needs-test-cases
+  - name: remove label needs-test-cases
+    conditions:
+      - label=needs-test-cases
+      - files~=^src/
+      - files~=^t/
+    actions:
+      label:
+        remove:
+          - needs-test-cases

.travis.yml

@@ -83,19 +83,19 @@ install:
   - git clone https://github.com/openresty/rds-json-nginx-module.git ../rds-json-nginx-module
   - git clone https://github.com/openresty/srcache-nginx-module.git ../srcache-nginx-module
   - git clone https://github.com/openresty/redis2-nginx-module.git ../redis2-nginx-module
-  - git clone https://github.com/openresty/lua-resty-core.git ../lua-resty-core
+  - git clone -b uri_not_comp https://github.com/zhuizhuhaomeng/lua-resty-core.git ../lua-resty-core
   - git clone https://github.com/openresty/lua-resty-lrucache.git ../lua-resty-lrucache
   - git clone https://github.com/openresty/lua-resty-mysql.git ../lua-resty-mysql
-  - git clone https://github.com/openresty/stream-lua-nginx-module.git ../stream-lua-nginx-module
+  - git clone -b uri_not_comp https://github.com/zhuizhuhaomeng/stream-lua-nginx-module.git ../stream-lua-nginx-module
   - git clone -b v2.1-agentzh https://github.com/openresty/luajit2.git luajit2
 
 before_script:
   - mysql -uroot -e 'create database ngx_test; grant all on ngx_test.* to "ngx_test"@"%" identified by "ngx_test"; flush privileges;'
 
 script:
   - sudo iptables -I OUTPUT 1 -p udp --dport 10086 -j REJECT
-  - sudo iptables -A OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP
-  - sudo iptables -A OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP
+  - sudo iptables -I OUTPUT -p tcp --dst 127.0.0.2 --dport 12345 -j DROP
+  - sudo iptables -I OUTPUT -p udp --dst 127.0.0.2 --dport 12345 -j DROP
   - cd luajit2/
   - make -j$JOBS CCDEBUG=-g Q= PREFIX=$LUAJIT_PREFIX CC=$CC XCFLAGS='-DLUA_USE_APICHECK -DLUA_USE_ASSERT -msse4.2' > build.log 2>&1 || (cat build.log && exit 1)
   - sudo make install PREFIX=$LUAJIT_PREFIX > build.log 2>&1 || (cat build.log && exit 1)

README.markdown

@@ -1512,13 +1512,17 @@ This hook is often used to create per-worker reoccurring timers (via the [ngx.ti
                  return
              end
          end
+
+         -- do something in timer
      end
 
      local hdl, err = new_timer(delay, check)
      if not hdl then
          log(ERR, "failed to create timer: ", err)
          return
      end
+
+     -- other job in init_worker_by_lua
  ';
 ```
 
@@ -4040,7 +4044,7 @@ in gzipped responses that cannot be handled properly in Lua code. Original reque
 
 When the `body` option is not specified and the `always_forward_body` option is false (the default value), the `POST` and `PUT` subrequests will inherit the request bodies of the parent request (if any).
 
-There is a hard-coded upper limit on the number of concurrent subrequests possible for every main request. In older versions of Nginx, the limit was `50` concurrent subrequests and in more recent versions, Nginx `1.1.x` onwards, this was increased to `200` concurrent subrequests. When this limit is exceeded, the following error message is added to the `error.log` file:
+There is a hard-coded upper limit on the number of subrequests possible for every main request. In older versions of Nginx, the limit was `50` concurrent subrequests and in more recent versions, Nginx `1.9.5` onwards, the same limit is changed to limit the depth of recursive subrequests. When this limit is exceeded, the following error message is added to the `error.log` file:
 
 
     [error] 13983#0: *1 subrequests cycle while processing "/uri"
@@ -4548,7 +4552,12 @@ or a Lua table holding the query arguments' key-value pairs, as in
  ngx.req.set_uri_args({ a = 3, b = "hello world" })
 ```
 
-where in the latter case, this method will escape argument keys and values according to the URI escaping rule.
+In the former case, i.e., when the whole query-string is provided directly,
+the input Lua string should already be well-formed with the URI encoding.
+For security considerations, this method will automatically escape any control and
+whitespace characters (ASCII code 0x00 ~ 0x32 and 0x7F) in the Lua string.
+
+In the latter case, this method will escape argument keys and values according to the URI escaping rule.
 
 Multi-value arguments are also supported:
 
@@ -4874,6 +4883,11 @@ ngx.req.set_header
 
 Set the current request's request header named `header_name` to value `header_value`, overriding any existing ones.
 
+The input Lua string `header_name` and `header_value` should already be well-formed with the URI encoding.
+For security considerations, this method will automatically escape " ", """, "(", ")", ",", "/", ":", ";", "?",
+"<", "=", ">", "?", "@", "[", "]", "\", "{", "}", 0x00-0x1F, 0x7F-0xFF in `header_name` and automatically escape
+"0x00-0x08, 0x0A-0x0F, 0x7F in `header_value`.
+
 By default, all the subrequests subsequently initiated by [ngx.location.capture](#ngxlocationcapture) and [ngx.location.capture_multi](#ngxlocationcapture_multi) will inherit the new header.
 
 Here is an example of setting the `Content-Type` header:
@@ -5567,11 +5581,15 @@ This method was introduced in the `0.5.0rc30` release.
 ngx.escape_uri
 --------------
 
-**syntax:** *newstr = ngx.escape_uri(str)*
+**syntax:** *newstr = ngx.escape_uri(str, type?)*
 
 **context:** *init_by_lua&#42;, init_worker_by_lua&#42;, set_by_lua&#42;, rewrite_by_lua&#42;, access_by_lua&#42;, content_by_lua&#42;, header_filter_by_lua&#42;, body_filter_by_lua&#42;, log_by_lua&#42;, ngx.timer.&#42;, balancer_by_lua&#42;, ssl_certificate_by_lua&#42;, ssl_session_fetch_by_lua&#42;, ssl_session_store_by_lua&#42;*
 
-Escape `str` as a URI component.
+Since `v0.10.16rc6`, this function accepts an optional `type` argument.
+When `type` is set to 0, escape `str` as a URI. And these
+characters " ", "#", "%", "?", 0x00-0x1F, 0x7F-0xFF will be escaped.
+When `type` is set to 2 (which is the default), escape `str`
+as a URI component. All characters excepter ALPHA, DIGIT, "-", ".", "_", "~" will be escaped.
 
 [Back to TOC](#nginx-api-for-lua)
 
@@ -6292,13 +6310,13 @@ When the `replace` is a string, then it is treated as a special template for str
 ```lua
 
  local newstr, n, err = ngx.re.sub("hello, 1234", "([0-9])[0-9]", "[$0][$1]")
- if newstr then
-     -- newstr == "hello, [12][1]34"
-     -- n == 1
- else
+ if not newstr then
      ngx.log(ngx.ERR, "error: ", err)
      return
  end
+
+ -- newstr == "hello, [12][1]34"
+ -- n == 1
 ```
 
 where `$0` referring to the whole substring matched by the pattern and `$1` referring to the first parenthesized capturing substring.
@@ -6308,17 +6326,17 @@ Curly braces can also be used to disambiguate variable names from the background
 ```lua
 
  local newstr, n, err = ngx.re.sub("hello, 1234", "[0-9]", "${0}00")
-     -- newstr == "hello, 100234"
-     -- n == 1
+ -- newstr == "hello, 100234"
+ -- n == 1
 ```
 
 Literal dollar sign characters (`$`) in the `replace` string argument can be escaped by another dollar sign, for instance,
 
 ```lua
 
  local newstr, n, err = ngx.re.sub("hello, 1234", "[0-9]", "$$")
-     -- newstr == "hello, $234"
-     -- n == 1
+ -- newstr == "hello, $234"
+ -- n == 1
 ```
 
 Do not use backlashes to escape dollar signs; it will not work as expected.
@@ -6330,9 +6348,10 @@ When the `replace` argument is of type "function", then it will be invoked with
  local func = function (m)
      return "[" .. m[0] .. "][" .. m[1] .. "]"
  end
+
  local newstr, n, err = ngx.re.sub("hello, 1234", "( [0-9] ) [0-9]", func, "x")
-     -- newstr == "hello, [12][1]34"
-     -- n == 1
+ -- newstr == "hello, [12][1]34"
+ -- n == 1
 ```
 
 The dollar sign characters in the return value of the `replace` function argument are not special at all.
@@ -6357,13 +6376,13 @@ Here is some examples:
 ```lua
 
  local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", "[$0,$1]", "i")
- if newstr then
-     -- newstr == "[hello,h], [world,w]"
-     -- n == 2
- else
+ if not newstr then
      ngx.log(ngx.ERR, "error: ", err)
      return
  end
+
+ -- newstr == "[hello,h], [world,w]"
+ -- n == 2
 ```
 
 ```lua
@@ -6372,8 +6391,8 @@ Here is some examples:
      return "[" .. m[0] .. "," .. m[1] .. "]"
  end
  local newstr, n, err = ngx.re.gsub("hello, world", "([a-z])[a-z]+", func, "i")
-     -- newstr == "[hello,h], [world,w]"
-     -- n == 2
+ -- newstr == "[hello,h], [world,w]"
+ -- n == 2
 ```
 
 This method requires the PCRE library enabled in Nginx ([Known Issue With Special Escaping Sequences](#special-escaping-sequences)).
@@ -6548,6 +6567,10 @@ The optional `flags` argument specifies a user flags value associated with the e
 
 When it fails to allocate memory for the current key-value item, then `set` will try removing existing items in the storage according to the Least-Recently Used (LRU) algorithm. Note that, LRU takes priority over expiration time here. If up to tens of existing items have been removed and the storage left is still insufficient (either due to the total capacity limit specified by [lua_shared_dict](#lua_shared_dict) or memory segmentation), then the `err` return value will be `no memory` and `success` will be `false`.
 
+If the sizes of items in the dictionary are not multiples or even powers of a certain value (like 2), it is easier to encounter `no memory` error because of memory fragmentation. It is recommended to use different dictionaries for different sizes of items.
+
+When you encounter `no memory` error, you can also evict more least-recently-used items by retrying this method call more times to to make room for the current item.
+
 If this method succeeds in storing the current item by forcibly removing other not-yet-expired items in the dictionary via LRU, the `forcible` return value will be `true`. If it stores the item without forcibly removing other valid items, then the return value `forcible` will be `false`.
 
 The first argument to this method must be the dictionary object itself, for example,
@@ -7074,6 +7097,9 @@ Since the `v0.7.18` release, connecting to a datagram unix domain socket file is
      ngx.say("failed to connect to the datagram unix domain socket: ", err)
      return
  end
+
+ -- do something after connect
+ -- such as sock:send or sock:receive
 ```
 
 assuming the datagram service is listening on the unix domain socket file `/tmp/some-datagram-service.sock` and the client socket will use the "autobind" feature on Linux.
@@ -7278,6 +7304,9 @@ Connecting to a Unix Domain Socket file is also possible:
      ngx.say("failed to connect to the memcached unix domain socket: ", err)
      return
  end
+
+ -- do something after connect
+ -- such as sock:send or sock:receive
 ```
 
 assuming memcached (or something else) is listening on the unix domain socket file `/tmp/memcached.sock`.
@@ -8156,6 +8185,8 @@ Here is a simple example:
              ngx.log(ngx.ERR, "failed to create timer: ", err)
              return
          end
+
+         -- other job in log_by_lua_block
      }
  }
 ```
@@ -8176,13 +8207,17 @@ One can also create infinite re-occurring timers, for instance, a timer getting
          ngx.log(ngx.ERR, "failed to create the timer: ", err)
          return
      end
+
+     -- do something in timer
  end
 
  local ok, err = ngx.timer.at(delay, handler)
  if not ok then
      ngx.log(ngx.ERR, "failed to create the timer: ", err)
      return
  end
+
+ -- do other jobs
 ```
 
 It is recommended, however, to use the [ngx.timer.every](#ngxtimerevery) API function