September 22, 2009 at 4:14 pm
· Filed under C10K, Software, nginx
Last Saturday I gave the talk “Nginx Internals” in Guangzhou. Here are the presentation slides and the video of the talk.
Nginx Internals Video part 1 (in Chinese):
Nginx Internals Video part 2 (in Chinese):
Nginx Internals Video part 3 (in Chinese):
Permalink
August 31, 2009 at 11:37 am
· Filed under C10K, Software, nginx

nginx map (click to view large image)
I’m going to give a free talk on nginx’s internals next month (September 19), in Guangzhou, China.
I’ve been reading the source code of nginx for a few days. Digging into this charming code is really a pleasant experience, though at first glance it appeared a little bit difficult to understand. Nginx becomes more and more popular, but unfortunately there is not enough documentation on its architecture and implementation. Now that I have spent a considerable amount of time reading the source code and have gained some knowledge, why not share it with those who want to know things under the hood?
So, if you are interested in this talk and you can be in Guangzhou that day, feel free to join in. Please comment on this post or drop me an email to let me know which parts you are interested in (see the mind map above, draft version though).
There might be a thousand Hamlets in a thousand people’s eyes. Note that I’m not Igor, and the only way I try to understand the nuts and bolts is by reverse engineering it, hence I can’t guarantee you no mistakes or misunderstandings in my talk. And frankly, it is not a trivial topic after all, not only because of the size of nginx’s code base, but also its elaborate design.
The speech will be in Chinese while slides will be in English. Specifics of time and location are coming soon. Stay tuned.
Update:
Time: 14:30-17:30, September 19, 2009
Location: Netease Building Tower E, Guangzhou Information Port #16 Keyun RD. Tianhe District, Guangzhou
Registration: http://blog.laiyonghao.com/2009/09/programming-tech-party/370
Permalink
August 13, 2009 at 3:53 pm
· Filed under Programming, nginx
Some of my friends think nginx modules are very difficult to write. Sure it’s not so easy but it’s not that hard either. The only problem is that the documentation is not enough. But don’t let this scare yourself away, the situation is improving.
So I write a hello world nginx module here. It’s pretty short and with enough comments. If you’re a new nginx module developer, feel free to take it as an example and replace hello with whatever your module name is, then start your happy nginx module hacking journey. And I’ll write more topics on nginx soon. Look for it!
ngx_http_hello_module.c:
/*
* Copyright (C) Joshua Zhu, http://www.zhuzhaoyuan.com
*/
#include <ngx_config.h>
#include <ngx_core.h>
#include <ngx_http.h>
static char *ngx_http_hello(ngx_conf_t *cf, ngx_command_t *cmd,
void *conf);
static ngx_command_t ngx_http_hello_commands[] = {
{ ngx_string("hello"),
NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS,
ngx_http_hello,
0,
0,
NULL },
ngx_null_command
};
static u_char ngx_hello_string[] = "Hello, world!";
static ngx_http_module_t ngx_http_hello_module_ctx = {
NULL, /* preconfiguration */
NULL, /* postconfiguration */
NULL, /* create main configuration */
NULL, /* init main configuration */
NULL, /* create server configuration */
NULL, /* merge server configuration */
NULL, /* create location configuration */
NULL /* merge location configuration */
};
ngx_module_t ngx_http_hello_module = {
NGX_MODULE_V1,
&ngx_http_hello_module_ctx, /* module context */
ngx_http_hello_commands, /* module directives */
NGX_HTTP_MODULE, /* module type */
NULL, /* init master */
NULL, /* init module */
NULL, /* init process */
NULL, /* init thread */
NULL, /* exit thread */
NULL, /* exit process */
NULL, /* exit master */
NGX_MODULE_V1_PADDING
};
static ngx_int_t
ngx_http_hello_handler(ngx_http_request_t *r)
{
ngx_int_t rc;
ngx_buf_t *b;
ngx_chain_t out;
/* we response to 'GET' and 'HEAD' requests only */
if (!(r->method & (NGX_HTTP_GET|NGX_HTTP_HEAD))) {
return NGX_HTTP_NOT_ALLOWED;
}
/* discard request body, since we don't need it here */
rc = ngx_http_discard_request_body(r);
if (rc != NGX_OK) {
return rc;
}
/* set the 'Content-type' header */
r->headers_out.content_type_len = sizeof("text/html") - 1;
r->headers_out.content_type.len = sizeof("text/html") - 1;
r->headers_out.content_type.data = (u_char *) "text/html";
/* send the header only, if the request type is http 'HEAD' */
if (r->method == NGX_HTTP_HEAD) {
r->headers_out.status = NGX_HTTP_OK;
r->headers_out.content_length_n = sizeof(ngx_hello_string) - 1;
return ngx_http_send_header(r);
}
/* allocate a buffer for your response body */
b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t));
if (b == NULL) {
return NGX_HTTP_INTERNAL_SERVER_ERROR;
}
/* attach this buffer to the buffer chain */
out.buf = b;
out.next = NULL;
/* adjust the pointers of the buffer */
b->pos = ngx_hello_string;
b->last = ngx_hello_string + sizeof(ngx_hello_string) - 1;
b->memory = 1; /* this buffer is in memory */
b->last_buf = 1; /* this is the last buffer in the buffer chain */
/* set the status line */
r->headers_out.status = NGX_HTTP_OK;
r->headers_out.content_length_n = sizeof(ngx_hello_string) - 1;
/* send the headers of your response */
rc = ngx_http_send_header(r);
if (rc == NGX_ERROR || rc > NGX_OK || r->header_only) {
return rc;
}
/* send the buffer chain of your response */
return ngx_http_output_filter(r, &out);
}
static char *
ngx_http_hello(ngx_conf_t *cf, ngx_command_t *cmd, void *conf)
{
ngx_http_core_loc_conf_t *clcf;
clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
clcf->handler = ngx_http_hello_handler; /* handler to process the 'hello' directive */
return NGX_CONF_OK;
}
config:
ngx_addon_name=ngx_http_hello_module
HTTP_MODULES="$HTTP_MODULES ngx_http_hello_module"
NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/ngx_http_hello_module.c"
hello.conf:
# hello
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen 8080;
server_name localhost;
location / {
hello;
}
}
}
If you want more detailed information, please consult Evan Miller’s Guide To Nginx Module Development.
Permalink
July 13, 2009 at 2:51 pm
· Filed under C10K, Programming, memcached
Memcached becomes more and more popular nowadays. It is widely used by many heavy loaded sites. Why does it succeed?
Well, of course the first and the most important reason is that it meets the need for speed of the web 2.0 sites, by caching data and objects in memory. However, from the point of view of a server developer, what I want to emphasize is that it is the simplicity of memcached’s protocol design makes it more successful. Take a look at memcached’s protocol:
storage: ("set", "add", "replace", "append", "prepend", "cas")
<command name> <key> <flags> <exptime> <bytes> [noreply]rn
cas <key> <flags> <exptime> <bytes> <cas unique> [noreply]rn
reply: ("ERRORrn", "CLIENT_ERROR <error>rn", "SERVER_ERROR <error>rn",
"STOREDrn", "NOT_FOUNDrn", "EXISTSrn", "NOT_FOUNDrn")
retrieval: ("get", "gets")
get <key>rn
gets <key>rn
reply: ("ENDrn",
"VALUE <key> <flags> <bytes> [<cas unique>]rn<data block>rn")
deletion:
delete <key> [<time>] [noreply]rn
reply: ("DELETEDrn", "NOT_FOUNDrn")
increment/decrement: ("incr", "decr")
incr <key> <value> [noreply]rn
decr <key> <value> [noreply]rn
reply: ("NOT_FOUNDrn",
"<value>rn")
statistics: ("stat")
statsrn
stats <args>rn
reply: ("STAT <name> <value>rn",
"STAT items:<slabclass>:<stat> <value>rn"
"ENDrn")
other:
flush_all
reply: ("OKrn")
versionrn
reply: ("VERSION <version>rn")
verbosity
reply: ("OKrn")
quit
With the textual protocol as shown above, memcache can be easily supported and implemented in various programming languages. No wonder dozens of different memcache clients appear. And then it consequently boosts memcached’s use. Simple thing usually will withstand the test of time. The old simple textual protocols, e.g., HTTP, FTP, SMTP and POP3 are still in use on the modern Internet. Not only because textual protocols can be easily parsed and extended, but also they are convenient for human being to read and debug. This is where the UNIX philosophy shines.
In conclusion, always prefer textual protocol when designing your own application. It would turn out to be really a wise decision.
Permalink
March 17, 2009 at 6:59 pm
· Filed under Software
For a long time, I was not satisfied with vmstat, because it does not generate timestamps. Then I came across a cool program named Dstat yesterday, which can be an excellent replacement for vmstat.

A screenshot of Dstat
By using Dstat, now not only all my needs can be met but also it is pretty easy to do resource usage analysis and graphing.
As the author described, “Dstat is a versatile replacement for vmstat, iostat, netstat, nfsstat and ifstat. Dstat overcomes some of their limitations and adds some extra features, more counters and flexibility. Dstat is handy for monitoring systems during performance tuning tests, benchmarks or troubleshooting.” So why use three or more tools when one tool can give you everything you need?
Here is an example showing how to use it.
First, capture the resource usage information (CPU and memory) to a file, e.g. stat.dat:
Then use the scripts below to create CPU and memory usage graphs:
#!/usr/bin/gnuplot
set terminal png
set output "cpu.png"
set title "CPU usage"
set xlabel "time"
set ylabel "percent"
set xdata time
set timefmt "%d-%m %H:%M:%S"
set format x "%H:%M"
plot "stat.dat" using 1:4 title "system" with lines,
"stat.dat" using 1:3 title "user" with lines,
"stat.dat" using 1:5 title "idle" with lines
#!/usr/bin/gnuplot
set terminal png
set output "memory.png"
set title "memory usage"
set xlabel "time"
set ylabel "size(M Bytes)"
set xdata time
set timefmt "%d-%m %H:%M:%S"
set format x "%H:%M"
plot "stat.dat" using 1:9 title "used" with lines,
"stat.dat" using 1:10 title "buff" with lines,
"stat.dat" using 1:11 title "cach" with lines,
"stat.dat" using 1:12 title "free" with lines
Resource usage graph examples:


Download the scripts.
http://www.zhuzhaoyuan.com/download/dstat/cpu.sh
http://www.zhuzhaoyuan.com/download/dstat/memory.sh
Permalink