A Mechanism to Help Write Web Application Firewalls for Nginx
Developing a Web Application Firewall module for Nginx is not an easy task. The lack of input body filters makes it harder. Nginx is an outstanding web server, but it is not perfect. Actually, nothing is perfect.
So we added the input body filter mechanism to our own Nginx distribution, which is named Tengine. By taking advantage of this mechanism, processing the request body is not that complicated anymore (In standard Nginx, request body may be buffered to disk file and you have to deal with up to two buffers)
Here I have an example to demonstrate how to write an input body filter. It is a simple module to fight hash collision DoS attacks.
/* * Copyright (C) Joshua Zhu, http://www.zhuzhaoyuan.com */ #include <ngx_config.h> #include <ngx_core.h> #include <ngx_http.h> typedef struct { ngx_flag_t enable; ngx_uint_t max_post_params; } ngx_http_anti_hashdos_loc_conf_t; typedef struct { ngx_uint_t post_params; } ngx_http_anti_hashdos_ctx_t; static ngx_int_t ngx_http_anti_hashdos_input_body_filter(ngx_http_request_t *r, ngx_buf_t *buf); static void *ngx_http_anti_hashdos_create_loc_conf(ngx_conf_t *cf); static char *ngx_http_anti_hashdos_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child); static ngx_int_t ngx_http_anti_hashdos_init(ngx_conf_t *cf); static ngx_command_t ngx_http_anti_hashdos_filter_commands[] = { { ngx_string("anti_hashdos"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, NGX_HTTP_LOC_CONF_OFFSET, offsetof(ngx_http_anti_hashdos_loc_conf_t, enable), NULL }, { ngx_string("anti_hashdos_max_post_params"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, ngx_conf_set_num_slot, NGX_HTTP_LOC_CONF_OFFSET, offsetof(ngx_http_anti_hashdos_loc_conf_t, max_post_params), NULL }, ngx_null_command }; static ngx_http_module_t ngx_http_anti_hashdos_filter_module_ctx = { NULL, /* preconfiguration */ ngx_http_anti_hashdos_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ ngx_http_anti_hashdos_create_loc_conf,/* create location configuration */ ngx_http_anti_hashdos_merge_loc_conf /* merge location configuration */ }; ngx_module_t ngx_http_anti_hashdos_filter_module = { NGX_MODULE_V1, &ngx_http_anti_hashdos_filter_module_ctx, /* module context */ ngx_http_anti_hashdos_filter_commands,/* module directives */ NGX_HTTP_MODULE, /* module type */ NULL, /* init master */ NULL, /* init module */ NULL, /* init process */ NULL, /* init thread */ NULL, /* exit thread */ NULL, /* exit process */ NULL, /* exit master */ NGX_MODULE_V1_PADDING }; static ngx_http_input_body_filter_pt ngx_http_next_input_body_filter; static ngx_int_t ngx_http_anti_hashdos_input_body_filter(ngx_http_request_t *r, ngx_buf_t *buf) { u_char *p; ngx_http_anti_hashdos_ctx_t *ctx; ngx_http_anti_hashdos_loc_conf_t *ahlf; ahlf = ngx_http_get_module_loc_conf(r, ngx_http_anti_hashdos_filter_module); if (!ahlf->enable) { return ngx_http_next_input_body_filter(r, buf); } ctx = ngx_http_get_module_ctx(r, ngx_http_anti_hashdos_filter_module); if (ctx == NULL) { ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_anti_hashdos_ctx_t)); if (ctx == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } ctx->post_params = 1; } for (p = buf->pos; p < buf->last; p++) { if (*p == '&') { ctx->post_params++; } } if (ctx->post_params > ahlf->max_post_params) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "anti hashdos: \"%V\" blocked, too many post params: %d", &r->connection->addr_text, ctx->post_params); return NGX_HTTP_BAD_REQUEST; } return ngx_http_next_input_body_filter(r, buf); } static void * ngx_http_anti_hashdos_create_loc_conf(ngx_conf_t *cf) { ngx_http_anti_hashdos_loc_conf_t *conf; conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_anti_hashdos_loc_conf_t)); if (conf == NULL) { return NULL; } conf->enable = NGX_CONF_UNSET; conf->max_post_params = NGX_CONF_UNSET_UINT; return conf; } static char * ngx_http_anti_hashdos_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child) { ngx_http_anti_hashdos_loc_conf_t *prev = parent; ngx_http_anti_hashdos_loc_conf_t *conf = child; ngx_conf_merge_value(conf->enable, prev->enable, 0); ngx_conf_merge_uint_value(conf->max_post_params, prev->max_post_params, 120); return NGX_CONF_OK; } static ngx_int_t ngx_http_anti_hashdos_init(ngx_conf_t *cf) { ngx_http_next_input_body_filter = ngx_http_top_input_body_filter; ngx_http_top_input_body_filter = ngx_http_anti_hashdos_input_body_filter; return NGX_OK; }
The code looks quite straight forward, right? And it is similar to an output body filter, just a few steps:
1) Implement your own input body filter function. e.g.
static ngx_int_t ngx_http_anti_hashdos_input_body_filter(ngx_http_request_t *r, ngx_buf_t *buf) { /* Do the input body filtering here */ }
2) In your input body filter function, return an HTTP error code if something is wrong. Otherwise, call ngx_http_next_input_body_filter(r, buf) directly to pass the buf to the next input body filters.
3) Install your input body filter in the post_configuration hook function. Push your input body filter to the head of the input body filter chain. e.g.
static ngx_int_t ngx_http_anti_hashdos_init(ngx_conf_t *cf) { ngx_http_next_input_body_filter = ngx_http_top_input_body_filter; ngx_http_top_input_body_filter = ngx_http_anti_hashdos_input_body_filter; return NGX_OK; }
NOTE: This is just a demonstration to show how to write input body filters. If you want to fight hash collision DoS attacks completely, you have to write more code and various POST content types should be processed.
Download the code here:
http://www.zhuzhaoyuan.com/download/tengine/anti_hashdos.tar.gz
james said,
January 18, 2012 @ 9:20 am
#define NGX_MAX_ALLOC_FROM_POOL (ngx_pagesize - 1)
请教，为什么是这样
Joshua said,
January 18, 2012 @ 9:50 am
@james,
大于PAGE_SIZE或pool的内存nginx会单独存放一个链表，这个是可能被回收的，以节约内存。请阅读src/core/ngx_palloc.c和我之前写的nginx internals文档中讲内存池实现部分
james said,
January 18, 2012 @ 10:57 am
我的意思是 比如linux的pagesize是 4k ，它为什么是4k-1
Joshua said,
January 18, 2012 @ 11:25 am
@james,
>= PAGE_SIZE的内存回收更有意义
Jason.Lee said,
January 18, 2012 @ 2:20 pm
不仅仅是佩服
tricky1997 said,
January 19, 2012 @ 6:15 pm
请问，为什么不通过入侵检测工具（例如snort）来防止hash Dos，而要在nginx里加功能呢？
Abioy said,
February 12, 2012 @ 10:21 am
Cool! Tengine把input filter给做了？我们某个服务由于已经在ACCESS PHASE中处理body，便直接在该PHASE加入了防御。
Joshua said,
February 12, 2012 @ 3:05 pm
@Abioy:
是的。在Nginx的phase里是无法完美处理body的，所以我们把input body filter加上了
Anders said,
February 19, 2012 @ 5:47 am
sorry for posting unrelated question here:
First of all, your blog has been very helpful for nginx newbies. Thanks a lot!
I am playing with your hello world module and attached it with NGX_HTTP_ACCESS_PHASE. As you may know, if you apply the basic auth on a uri that is backed by a remote resource through reversed proxy, the authentication header is carried over and hit the remote resource. For lots of cases, this will invoke the auth mechanism of the remote resource, say tomcat.
I understand you can bypass it by proxy_set_header Authorization “”; in the config file. But I am wondering if there is a way to suppress/remove it from the header_in in my module?
I tried: r->headers_in.authorization->value.len=0;
It appears to work. But I don’t know this could lead any memory leak since the authorization is a pointer and probably the memory is dynamically allocated for the request.
Could you shed some light here?
Thxs!
Anders
Jasmine said,
February 20, 2012 @ 7:32 am
You’d also have to egnarle your limits in /etc/limits.conf (or where ever it is on your distro of choice), if you’d have 1000s of users simultaneous, you could have a bottleneck if there are too many files open.
sfjnhi said,
February 21, 2012 @ 2:14 am
eKQS6X mbawurlovlyy
kgljmlz said,
February 21, 2012 @ 9:19 pm
PtHOGM , [url=http://mvgmtqvcpbff.com/]mvgmtqvcpbff[/url], [link=http://bdchmszfnucr.com/]bdchmszfnucr[/link], http://pbblsvrjlcsh.com/