PlumeLog簡介

無入侵的分布式日志系統(tǒng)，基于log4j、log4j2、logback搜集日志，設(shè)置鏈路ID，方便查詢關(guān)聯(lián)日志
基于elasticsearch作為查詢引擎
高吞吐，查詢效率高
全程不占應(yīng)用程序本地磁盤空間，免維護(hù);對于項(xiàng)目透明，不影響項(xiàng)目本身運(yùn)行
無需修改老項(xiàng)目，引入直接使用，支持dubbo,支持springcloud

服務(wù)端安裝

首先是消息隊(duì)列，PlumeLog適配了redis或kafka，一般項(xiàng)目redis足夠了，我這邊也是直接用redis，redis 官網(wǎng)：https://redis.io
然后需要安裝elasticsearch，官網(wǎng)下載地址:https://www.elastic.co/cn/downloads/past-releases
最后下載Plumelog的程序包，plumelog-server，下載地址：https://gitee.com/frankchenlong/plumelog/releases

這邊為了方便直接用docker部署redis和es

redis運(yùn)行腳本

docker run --name redis6379 -d \
-p 6379:6379 \
-v `pwd`/data:/data \
redis

es執(zhí)行腳本

docker run --name elasticsearch -d \
-e ES_JAVA_OPTS="-Xms512m -Xmx512m" \
-e "discovery.type=single-node" \
-p 9200:9200 \
-p 9300:9300 \
elasticsearch:7.7.0

啟動(dòng)服務(wù)

修改配置

下載好的Plumelog上傳到服務(wù)器，配置信息修改為自己的服務(wù)器地址

spring.application.name=plumelog_server
server.port=8891
spring.thymeleaf.mode=LEGACYHTML5
spring.mvc.view.prefix=classpath:/templates/
spring.mvc.view.suffix=.html
spring.mvc.static-path-pattern=/plumelog/**

#值為4種 redis,kafka,rest,restServer
#redis 表示用redis當(dāng)隊(duì)列
#kafka 表示用kafka當(dāng)隊(duì)列
#rest 表示從rest接口取日志
#restServer 表示作為rest接口服務(wù)器啟動(dòng)
#ui 表示單獨(dú)作為ui啟動(dòng)
plumelog.model=redis

#如果使用kafka,啟用下面配置
#plumelog.kafka.kafkaHosts=172.16.247.143:9092,172.16.247.60:9092,172.16.247.64:9092
#plumelog.kafka.kafkaGroupName=logConsumer


plumelog.queue.redis.redisHost=127.0.0.1:6379
#如果使用redis有密碼,啟用下面配置
#plumelog.queue.redis.redisPassWord=plumelogredis
#plumelog.queue.redis.redisDb=0

#redis配置,3.0版本必須配置redis地址，因?yàn)樾枰O(jiān)控報(bào)警
plumelog.redis.redisHost=127.0.0.1:6379
#如果使用redis有密碼,啟用下面配置
#plumelog.redis.redisPassWord=plumelogredis

#如果使用rest,啟用下面配置
#plumelog.rest.restUrl=http://127.0.0.1:8891/getlog
#plumelog.rest.restUserName=plumelog
#plumelog.rest.restPassWord=123456

#elasticsearch相關(guān)配置
plumelog.es.esHosts=127.0.0.1:9200
#ES7.*已經(jīng)去除了索引type字段，所以如果是es7不用配置這個(gè)，7.*以下不配置這個(gè)會(huì)報(bào)錯(cuò)
#plumelog.es.indexType=plumelog
plumelog.es.shards=5
plumelog.es.replicas=1
plumelog.es.refresh.interval=30s
#日志索引建立方式day表示按天、hour表示按照小時(shí)
plumelog.es.indexType.model=day
#ES設(shè)置密碼,啟用下面配置
#plumelog.es.userName=elastic
#plumelog.es.passWord=DQNmQNVxpgYDotd0uKNW

#單次拉取日志條數(shù)
plumelog.maxSendSize=100
#拉取時(shí)間間隔，kafka不生效
plumelog.interval=100

#plumelog-ui的地址 如果不配置，報(bào)警信息里不可以點(diǎn)連接
plumelog.ui.url=https://127.0.0.1:8891

#管理密碼，手動(dòng)刪除日志的時(shí)候需要輸入的密碼
admin.password=123456
#日志保留天數(shù),配置0或者不配置默認(rèn)永久保留
admin.log.keepDays=30

#鏈路保留天數(shù),配置0或者不配置默認(rèn)永久保留
admin.log.trace.keepDays=30

login.username=admin
login.password=admin

啟動(dòng)plumelog，文件中我們可以看到有不同系統(tǒng)的執(zhí)行腳本，我是linux系統(tǒng)，所以直接執(zhí)行startup.sh

創(chuàng)建SpringBoot項(xiàng)目

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.3</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>org.jeemp</groupId>
    <artifactId>jeemp-plumelog</artifactId>
    <version>1.0.0</version>
    <name>jeemp-plumelog</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>2.1.4</version>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <!--分布式日志收集plumelog-->
        <dependency>
            <groupId>com.plumelog</groupId>
            <artifactId>plumelog-logback</artifactId>
            <version>3.3</version>
        </dependency>
        <dependency>
            <groupId>com.plumelog</groupId>
            <artifactId>plumelog-trace</artifactId>
            <version>3.3</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
            <version>2.1.11.RELEASE</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>cn.hutool</groupId>
            <artifactId>hutool-core</artifactId>
            <version>5.5.8</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

application.properties

server.port=8888
spring.datasource.url=jdbc:mysql://172.16.10.54:3306/jeemp?useUnicode=true&characterEncoding=utf-8&allowMultiQueries=true&useSSL=false
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.hikari.maximum-pool-size=10
# mybatis配置
mybatis.type-aliases-package=org.jeemp.plumelog.model
mybatis.configuration.map-underscore-to-camel-case=true
mybatis.configuration.default-fetch-size=100
mybatis.configuration.default-statement-timeout=30
# jackson格式化日期時(shí)間
spring.jackson.date-format=YYYY-MM-dd HH:mm:ss
spring.jackson.time-zone=GMT+8
spring.jackson.serialization.write-dates-as-timestamps=false
# sql日志打印
logging.level.com.example.demomybatis.dao=debug
## 輸出的日志文件
logging.file.path=/log

logback.xml

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <appender name="consoleApp" class="ch.qos.logback.core.ConsoleAppender">
        <layout class="ch.qos.logback.classic.PatternLayout">
            <pattern>
                %date{yyyy-MM-dd HH:mm:ss.SSS} %-5level[%thread]%logger{56}.%method:%L -%msg%n
            </pattern>
        </layout>
    </appender>

    <appender name="fileInfoApp" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <filter class="ch.qos.logback.classic.filter.LevelFilter">
            <level>ERROR</level>
            <onMatch>DENY</onMatch>
            <onMismatch>ACCEPT</onMismatch>
        </filter>
        <encoder>
            <pattern>
                %date{yyyy-MM-dd HH:mm:ss.SSS} %-5level[%thread]%logger{56}.%method:%L -%msg%n
            </pattern>
        </encoder>
        <!-- 滾動(dòng)策略 -->
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- 路徑 -->
            <fileNamePattern>log/info.%d.log</fileNamePattern>
        </rollingPolicy>
    </appender>

    <appender name="fileErrorApp" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>ERROR</level>
        </filter>
        <encoder>
            <pattern>
                %date{yyyy-MM-dd HH:mm:ss.SSS} %-5level[%thread]%logger{56}.%method:%L -%msg%n
            </pattern>
        </encoder>

        <!-- 設(shè)置滾動(dòng)策略 -->
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <!-- 路徑 -->
            <fileNamePattern>log/error.%d.log</fileNamePattern>

            <!-- 控制保留的歸檔文件的最大數(shù)量，超出數(shù)量就刪除舊文件，假設(shè)設(shè)置每個(gè)月滾動(dòng)，
            且<maxHistory> 是1，則只保存最近1個(gè)月的文件，刪除之前的舊文件 -->
            <MaxHistory>1</MaxHistory>
        </rollingPolicy>
    </appender>

    <appender name="plumelog" class="com.plumelog.logback.appender.RedisAppender">
        <appName>mybatisDemo</appName>
        <redisHost>172.16.10.54</redisHost>
        <redisAuth></redisAuth>
        <redisPort>6379</redisPort>
        <runModel>2</runModel>
    </appender>

    <!-- root 一定要放在最后，因有加載順序的問題 -->
    <root level="INFO">
        <appender-ref ref="consoleApp"/>
        <appender-ref ref="fileInfoApp"/>
        <appender-ref ref="fileErrorApp"/>
        <appender-ref ref="plumelog"/>
    </root>
</configuration>

啟動(dòng)項(xiàng)目

默認(rèn)用戶名和密碼都是admin

SpringBoot集成plumelog日志系統(tǒng)

PlumeLog簡介

服務(wù)端安裝

修改配置

創(chuàng)建SpringBoot項(xiàng)目

啟動(dòng)項(xiàng)目