Rootop 服务器运维与web架构

nginx日志转为json格式

# nginx配置文件

# 手动拼成json格式
log_format log_json
'{'
	'"remote_addr":"$remote_addr",'
	'"x-forwarded-for":"$http_x_forwarded_for",'
	'"remote_user":"$remote_user",'
	'"time_local":"$time_local",'
	'"request":"$request",'
	'"status":"$status",'
	'"body_bytes_sent":"$body_bytes_sent",'
	'"http_referer":"$http_referer",'
	'"user_agent":"$http_user_agent"'
'}';

# 通过内置参数实现,需要nginx版本 >= 1.11.8
# 官方文档:http://nginx.org/en/docs/http/ngx_http_log_module.html#log_format
log_format log_json2 escape=json
'{'
	'"remote_addr":"$remote_addr",'
	'"x-forwarded-for":"$http_x_forwarded_for",'
	'"remote_user":"$remote_user",'
	'"time_local":"$time_local",'
	'"request":"$request",'
	'"status":"$status",'
	'"body_bytes_sent":"$body_bytes_sent",'
	'"http_referer":"$http_referer",'
	'"user_agent":"$http_user_agent"'
'}';

server
{
	listen 80;
	root html;
	access_log /tmp/access.log log_json2;
}

# 浏览器请求地址
http://192.168.10.17/index.php?method=\add&id=1

# 不加 escape=json 参数日志打印

{"remote_addr":"192.168.1.139","x-forwarded-for":"-","remote_user":"-","time_local":"30/Aug/2019:14:28:46 +0800","request":"GET /index.php?method=\x5Cadd&id=1 HTTP/1.1","status":"404","body_bytes_sent":"555","http_referer":"-","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3314.0 Safari/537.36 SE 2.X MetaSr 1.0"}

# 添加 escape=json 参数日志打印

{"remote_addr":"192.168.1.139","x-forwarded-for":"","remote_user":"","time_local":"30/Aug/2019:14:28:24 +0800","request":"GET /index.php?method=\\add&id=1 HTTP/1.1","status":"404","body_bytes_sent":"555","http_referer":"","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3314.0 Safari/537.36 SE 2.X MetaSr 1.0"}

可以看到x-forwarded-for和request两个字段是不同的。
日志如果获取不到调用变量的值,则用 横杠 “-” 做为值,添加了escape=json参数则变为空字符串。
反斜杠 “\” 变为 “\x5C”,添加了escape=json参数则自动转义,前面又添加了个反斜杠 “\\”

之前手动拼出来的json格式,在python里解析时候就报错,知道是转义问题,但是不知道怎么解决,今天发现支持内部转义。
这样json格式便于日志分析处理。

原创文章,转载请注明。本文链接地址: https://www.rootop.org/pages/4528.html

作者:Venus

服务器运维与性能优化

评论已关闭。