<kbd id="afajh"><form id="afajh"></form></kbd>
<strong id="afajh"><dl id="afajh"></dl></strong>
    <del id="afajh"><form id="afajh"></form></del>
        1. <th id="afajh"><progress id="afajh"></progress></th>
          <b id="afajh"><abbr id="afajh"></abbr></b>
          <th id="afajh"><progress id="afajh"></progress></th>

          RESTful API 設(shè)計(jì)最佳實(shí)踐

          共 4761字,需瀏覽 10分鐘

           ·

          2022-01-20 14:01

          在最近的一次項(xiàng)目性能優(yōu)化過(guò)程中,通過(guò)火焰圖工具發(fā)現(xiàn)logback占用CPU很多,因此有了這篇總結(jié)文章。

          logback 同步 vs 異步

          同步寫日志一般配置如下:

          1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
           <appender name="ORDER_LOG" class="ch.qos.logback.core.rolling.RollingFileAppender">
          <file>/data/logs/AAA/order.logfile>

          <encoder>
          <pattern>[%-5p] [%d{yyyy-MM-dd HH:mm:ss.SSS}] [%X{tracing_id}] [%C{1}:%M:%L] %m%npattern>
          <immediateFlush>falseimmediateFlush>
          encoder>
          <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
          <level>INFOlevel>
          filter>
          <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
          <fileNamePattern>/data/logs/AAA/order.log.%d{yyyy-MM-dd_HH}fileNamePattern>
          rollingPolicy>
          appender>


          從配置可以看出來(lái),寫日志到日志文件的操作由RollingFileAppender完成,該類的繼承結(jié)構(gòu)如下:

          從繼承結(jié)構(gòu)可以知道RollingFileAppender繼承了UnsynchronizedAppenderBase,根據(jù)doc文檔說(shuō)明,RollingFileAppender需要自己處理多線程同步的問(wèn)題。在內(nèi)部它確實(shí)也自己做了同步。

          RollingFileAppender
          1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          @Override
          protected void subAppend(E event) {
          // The roll-over check must precede actual writing. This is the
          // only correct behavior for time driven triggers.

          // We need to synchronize on triggeringPolicy so that only one rollover
          // occurs at a time
          synchronized (triggeringPolicy) {
          if (triggeringPolicy.isTriggeringEvent(currentlyActiveFile, event)) {
          rollover();
          }
          }
          super.subAppend(event);
          }

          OutputStreamAppender.java

          OutputStreamAppender.java
          1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
          31
          /**
          * All synchronization in this class is done via the lock object.
          */
          protected LogbackLock lock = new LogbackLock();

          protected void subAppend(E event) {
          if (!isStarted()) {
          return;
          }
          try {
          // this step avoids LBCLASSIC-139
          if (event instanceof DeferredProcessingAware) {
          ((DeferredProcessingAware) event).prepareForDeferredProcessing();
          }
          // the synchronization prevents the OutputStream from being closed while we
          // are writing. It also prevents multiple threads from entering the same
          // converter. Converters assume that they are in a synchronized block.
          synchronized (lock) {
          writeOut(event);
          }
          } catch (IOException ioe) {
          // as soon as an exception occurs, move to non-started state
          // and add a single ErrorStatus to the SM.
          this.started = false;
          addStatus(new ErrorStatus("IO failure in appender", this, ioe));
          }
          }
          // 通過(guò) encode進(jìn)行日志序列化,格式化寫入
          protected void writeOut(E event) throws IOException {
          this.encoder.doEncode(event);
          }

          LayoutWrappingEncoder.java

          1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          LayoutWrappingEncoder.java
          private boolean immediateFlush = true;
          public void doEncode(E event) throws IOException {
          String txt = layout.doLayout(event);
          outputStream.write(convertToBytes(txt));
          //是否立即寫入
          if (immediateFlush)
          outputStream.flush();
          }
          }

          通過(guò)上面的代碼邏輯可以得出如下結(jié)論:

          1. RollingFileAppender 寫日志自己實(shí)現(xiàn)了多線程同步。

          2. RollingFileAppender 寫日志是直接接入到日志文件中的。

          3. RollingFileAppender 默認(rèn)日志到文件后會(huì)立刻flush,保證日志不丟失。

          4. 因?yàn)槭琼樞驅(qū)懳募俣冗€是很高的,但是為每次都flush,這會(huì)影響性能。

          為了防止寫日志影響應(yīng)用性能, 我們需要使用異步寫日志的方式。

          異步寫日志一般配置如下:

          1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          <appender name ="ASYNC_ORDER_LOG" class= "ch.qos.logback.classic.AsyncAppender">
          // 不丟棄日志
          <discardingThreshold>0discardingThreshold>

          <queueSize>512queueSize>
          <includeCallerData>trueincludeCallerData>
          // 如果設(shè)置為true,隊(duì)列滿了會(huì)直接丟棄信息,而不是阻塞(其實(shí)就是使用的offer而不是put方法)
          <neverBlock>falseneverBlock>
          // 指定底層真實(shí)使用的Appender
          <appender-ref ref ="ORDER_LOG"/>
          appender>

          AsyncAppender?繼承自?AsyncAppenderBase。?AsyncAppenderBase的doc文檔有如下的描述:

          AsyncAppenderBase的子類寫日志是異步的方式,內(nèi)部使用了 BlockingQueue(異步寫日志就是一個(gè)生產(chǎn)者-消費(fèi)者模式)。
          BlockingQueue 的使用者負(fù)責(zé)在應(yīng)用關(guān)閉時(shí)關(guān)閉BlockingQueue,來(lái)確保不丟失日志。

          注意:不丟失日志,可以通過(guò)如下的方式實(shí)現(xiàn):

          1
          2
          3
          <configuration debug="true">
          <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook" />
          configuration>

          1
          2
          3
          4
          Runtime.addShutdownHook(new Thread (() -> {
          LoggerContext loggerContext = (LoggerContext) LoggerFactory.getILoggerFactory();
          loggerContext.stop();
          }));

          AsyncAppenderBase代碼分析:

          1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
          31
          32
          33
          34
          35
          36
          37
          38
          39
          40
          41
          42
          43
          44
          45
          46
          47
          48
          49
          50
          51
          52
          53
          54
          55
          56
          57
          58
          59
          60
          61
          62
          63
          64
          65
          66
          67
          68
          69
          70
          71
          72
          73
          74
          75
          76
          77
          78
          79
          80
          81
          82
          83
          84
          85
          86
          87
          88
          89
          90
          91
          public static final int DEFAULT_QUEUE_SIZE = 256;
          // 默認(rèn)阻塞隊(duì)列大小
          int queueSize = DEFAULT_QUEUE_SIZE;
          static final int UNDEFINED = -1;
          // 初始值為 -1, 在啟動(dòng)時(shí)默認(rèn)值是通過(guò)queueSize的大小計(jì)算出來(lái)的。
          int discardingThreshold = UNDEFINED;

          // 異步寫日志的線程
          Worker worker = new Worker();

          @Override
          public void start() {
          if (appenderCount == 0) {
          addError("No attached appenders found.");
          return;
          }
          if (queueSize < 1) {
          addError("Invalid queue size [" + queueSize + "]");
          return;
          }
          blockingQueue = new ArrayBlockingQueue(queueSize);

          if (discardingThreshold == UNDEFINED)
          discardingThreshold = queueSize / 5;
          addInfo("Setting discardingThreshold to " + discardingThreshold);
          worker.setDaemon(true);
          worker.setName("AsyncAppender-Worker-" + worker.getName());
          // make sure this instance is marked as "started" before staring the worker Thread
          super.start();
          worker.start();
          }

          @Override
          protected void append(E eventObject) {
          // 可以的隊(duì)列容量小于配置的值 并且 日志事件的級(jí)別為INFO及一下,那么就丟棄日志。
          if (isQueueBelowDiscardingThreshold() && isDiscardable(eventObject)) {
          return;
          }
          preprocess(eventObject);
          put(eventObject);
          }

          private boolean isQueueBelowDiscardingThreshold() {
          return (blockingQueue.remainingCapacity() < discardingThreshold);
          }

          protected boolean isDiscardable(ILoggingEvent event) {
          Level level = event.getLevel();
          return level.toInt() <= Level.INFO_INT;
          }

          protected void preprocess(ILoggingEvent eventObject) {
          eventObject.prepareForDeferredProcessing();
          // 是否需要獲取調(diào)用者信息,這也是一個(gè)耗時(shí)操作,默認(rèn)是false(內(nèi)部通過(guò)創(chuàng)建異常對(duì)象,獲取堆棧信息,從而計(jì)算日志發(fā)送的類,方法,行號(hào)信息)。
          if(includeCallerData)
          eventObject.getCallerData();
          }

          private void put(E eventObject) {
          try {
          // 阻塞操作
          blockingQueue.put(eventObject);
          } catch (InterruptedException e) {
          }
          }

          class Worker extends Thread {

          public void run() {
          AsyncAppenderBase parent = AsyncAppenderBase.this;
          AppenderAttachableImpl aai = parent.aai;

          // loop while the parent is started
          while (parent.isStarted()) {
          try {
          // 從隊(duì)列中獲取數(shù)據(jù),其實(shí)就是消費(fèi)者
          E e = parent.blockingQueue.take();
          aai.appendLoopOnAppenders(e);
          } catch (InterruptedException ie) {
          break;
          }
          }

          addInfo("Worker thread will flush remaining events before exiting. ");
          for (E e : parent.blockingQueue) {
          aai.appendLoopOnAppenders(e);
          }

          aai.detachAndStopAllAppenders();
          }
          }

          小結(jié)

          1. 同步寫日志而且immediateFlush=true的配置下,性能最差。原因是寫磁盤導(dǎo)致其他線程等待時(shí)間過(guò)長(zhǎng),雖然是順序?qū)懀钱吘故浅志没瘮?shù)據(jù)到磁盤。當(dāng)然了,SSD能好一點(diǎn)。

          2. 異步寫在JVM突然crash的時(shí)候有丟失數(shù)據(jù)的風(fēng)險(xiǎn),但是性能很高,原因在于避免了直接寫磁盤帶來(lái)的性能消耗。但是需要注意的是多線程操作同一個(gè)阻塞隊(duì)列也會(huì)因?yàn)殒i爭(zhēng)搶的問(wèn)題影響性能。
            a. 不同的模塊配置不同的日志文件和Appender,能減少鎖爭(zhēng)搶的問(wèn)題。
            b. 減少不必要的日志輸出。
            c. 增加阻塞隊(duì)列的大小,在
            neverBlock=false的情況下避免線程等待問(wèn)題。
            d. 多個(gè)Appender(SiftingAppender),底層還是寫同一個(gè)文件。好處是減少了多線程在阻塞隊(duì)列上的鎖競(jìng)爭(zhēng)問(wèn)題。

          SiftingAppender

          SiftingAppender是logback根據(jù)mdc中的變量動(dòng)態(tài)創(chuàng)建appender的代理,只要我們將一個(gè)線程號(hào)作為日志名分發(fā)器discriminator注入到SiftingAppender中,它就可以動(dòng)態(tài)的為我們創(chuàng)建不同的appender,達(dá)到分線程的目的,配置方式舉例如下:


          <appender name="frameworkthread" class="ch.qos.logback.classic.sift.SiftingAppender">
          <discriminator class="ThreadDiscriminator">
          <key>threadNamekey>

          discriminator>
          <sift>
          <appender name="FILE-${threadName}" class="ch.qos.logback.core.rolling.RollingFileAppender">
          <encoder>
          <Encoding>UTF-8Encoding>
          <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS}[%c][%thread][%X{tradeNo}][%p]-%m%npattern>
          encoder>
          <rollingPolicy
          class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">

          <fileNamePattern>D:/test/threadlogs/${threadName}-%d{yyyy-MM-dd}.%i.log
          fileNamePattern>
          <maxFileSize>100MBmaxFileSize>
          <maxHistory>60maxHistory>
          <totalSizeCap>20GBtotalSizeCap>
          rollingPolicy>
          appender>
          sift>
          appender>


          source:? //leokongwq.github.io/2019/12/14/logback-best-practise.html

          喜歡,在看

          瀏覽 69
          點(diǎn)贊
          評(píng)論
          收藏
          分享

          手機(jī)掃一掃分享

          分享
          舉報(bào)
          評(píng)論
          圖片
          表情
          推薦
          點(diǎn)贊
          評(píng)論
          收藏
          分享

          手機(jī)掃一掃分享

          分享
          舉報(bào)
          <kbd id="afajh"><form id="afajh"></form></kbd>
          <strong id="afajh"><dl id="afajh"></dl></strong>
            <del id="afajh"><form id="afajh"></form></del>
                1. <th id="afajh"><progress id="afajh"></progress></th>
                  <b id="afajh"><abbr id="afajh"></abbr></b>
                  <th id="afajh"><progress id="afajh"></progress></th>
                  国产激情婷婷 | 国产激情小视频在线观看 | 欧美精品乱码99久久蜜桃 | 淫色综合网 | 操死我视频 |