myselect基于SQL的日志分析工具

myselect用sql語(yǔ)法對(duì)日志文件進(jìn)行統(tǒng)計(jì)分析，把要分析的日志文件當(dāng)成一個(gè)數(shù)據(jù)庫(kù)，里面的日志行當(dāng)作數(shù)據(jù)庫(kù)記錄，比awk等工具使用更方便

$ myselect -h 
usage: 
myselect 'sql sentence'; 用 sql進(jìn)行統(tǒng)計(jì)分析 
myselect -s 'log line';對(duì)日志行按空格進(jìn)行分割編號(hào) 
myselect -n 'log line' 'sql sentence'; 對(duì)日志行用sql進(jìn)行解析 
myselect -p 'sql sentence'; 查看sql語(yǔ)法解析結(jié)果 
myselect -c 'sql sentence'; 查看sql計(jì)算過(guò)程

對(duì)于如下的nginx日志

198.52.103.14 - - [29/Jun/2014:00:17:11 +0800] "GET /q/1403060495509100 HTTP/1.1" 200 26788 "http://wenda.so.com/q/1403060495509100" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727)" 221 0.532

如果要知道來(lái)源ip最多的是哪些，myselect實(shí)現(xiàn)如下

$ myselect 'select count($1),$1 from accesstest.log group by $1 order by count($1) desc limit 10' 
14 111.13.65.251 
13 10.141.88.248 
12 10.141.88.239 
10 10.141.88.250 
9 121.226.135.115 
8 10.141.88.241 
8 10.141.88.249 
8 222.74.246.190 
7 211.149.165.150 
6 61.174.51.174