Spring Boot + Redis 搞定搜索欄熱搜、不雅文字過濾功能

來源：csdn.net/qq_25838777/article/details/109489767

使用java和redis實現(xiàn)一個簡單的熱搜功能，具備以下功能：

1. 搜索欄展示當(dāng)前登陸的個人用戶的搜索歷史記錄，刪除個人歷史記錄
2. 用戶在搜索欄輸入某字符，則將該字符記錄下來 以zset格式存儲的redis中，記錄該字符被搜索的個數(shù)以及當(dāng)前的時間戳 （用了DFA算法，感興趣的自己百度學(xué)習(xí)吧）
3. 每當(dāng)用戶查詢了已在redis存在了的字符時，則直接累加個數(shù)， 用來獲取平臺上最熱查詢的十條數(shù)據(jù)。（可以自己寫接口或者直接在redis中添加一些預(yù)備好的關(guān)鍵詞）
4. 最后還要做不雅文字過濾功能。這個很重要不說了你懂的。

代碼實現(xiàn)熱搜與個人搜索記錄功能，主要controller層下幾個方法就行了 ：

1. 向redis 添加熱搜詞匯(添加的時候使用下面不雅文字過濾的方法來過濾下這個詞匯，合法再去存儲
2. 每次點擊給相關(guān)詞熱度 +1
3. 根據(jù)key搜索相關(guān)最熱的前十名
4. 插入個人搜索記錄
5. 查詢個人搜索記錄

首先配置好redis數(shù)據(jù)源等等基礎(chǔ)

最后貼上核心的 服務(wù)層的代碼 ：

package?com.****.****.****.user;
?
import?com.jianlet.service.user.RedisService;
import?org.apache.commons.lang.StringUtils;
import?org.springframework.data.redis.core.*;
import?org.springframework.stereotype.Service;
import?javax.annotation.Resource;
import?java.util.*;
import?java.util.concurrent.TimeUnit;
?
/**
?*?@author:?mrwanghc
?*?@date:?2020/5/13
?*?@description:
?*/
@Transactional
@Service("redisService")
public?class?RedisServiceImpl?implements?RedisService?{
?
????//導(dǎo)入數(shù)據(jù)源
????@Resource(name?=?"redisSearchTemplate")
????private?StringRedisTemplate?redisSearchTemplate;
?
?
????//新增一條該userid用戶在搜索欄的歷史記錄
????//searchkey?代表輸入的關(guān)鍵詞
????@Override
????public?int?addSearchHistoryByUserId(String?userid,?String?searchkey)?{
????????String?shistory?=?RedisKeyUtils.getSearchHistoryKey(userid);
????????boolean?b?=?redisSearchTemplate.hasKey(shistory);
????????if?(b)?{
????????????Object?hk?=?redisSearchTemplate.opsForHash().get(shistory,?searchkey);
????????????if?(hk?!=?null)?{
????????????????return?1;
????????????}else{
????????????????redisSearchTemplate.opsForHash().put(shistory,?searchkey,?"1");
????????????}
????????}else{
????????????redisSearchTemplate.opsForHash().put(shistory,?searchkey,?"1");
????????}
????????return?1;
????}
?
????//刪除個人歷史數(shù)據(jù)
????@Override
????public?Long?delSearchHistoryByUserId(String?userid,?String?searchkey)?{
????????String?shistory?=?RedisKeyUtils.getSearchHistoryKey(userid);
????????return?redisSearchTemplate.opsForHash().delete(shistory,?searchkey);
????}
?
????//獲取個人歷史數(shù)據(jù)列表
????@Override
????public?List?getSearchHistoryByUserId(String?userid)?{
????????List?stringList?=?null;
????????String?shistory?=?RedisKeyUtils.getSearchHistoryKey(userid);
????????boolean?b?=?redisSearchTemplate.hasKey(shistory);
????????if(b){
????????????Cursor>?cursor?=?redisSearchTemplate.opsForHash().scan(shistory,?ScanOptions.NONE);
????????????while?(cursor.hasNext())?{
????????????????Map.Entry?map?=?cursor.next();
????????????????String?key?=?map.getKey().toString();
????????????????stringList.add(key);
????????????}
????????????return?stringList;
????????}
????????return?null;
????}
?
????//新增一條熱詞搜索記錄，將用戶輸入的熱詞存儲下來
????@Override
????public?int?incrementScoreByUserId(String?searchkey)?{
????????Long?now?=?System.currentTimeMillis();
????????ZSetOperations?zSetOperations?=?redisSearchTemplate.opsForZSet();
????????ValueOperations?valueOperations?=?redisSearchTemplate.opsForValue();
????????List?title?=?new?ArrayList<>();
????????title.add(searchkey);
????????for?(int?i?=?0,?lengh?=?title.size();?i?????????????String?tle?=?title.get(i);
????????????try?{
????????????????if?(zSetOperations.score("title",?tle)?<=?0)?{
????????????????????zSetOperations.add("title",?tle,?0);
????????????????????valueOperations.set(tle,?String.valueOf(now));
????????????????}
????????????}?catch?(Exception?e)?{
????????????????zSetOperations.add("title",?tle,?0);
????????????????valueOperations.set(tle,?String.valueOf(now));
????????????}
????????}
????????return?1;
????}
????
????//根據(jù)searchkey搜索其相關(guān)最熱的前十名?(如果searchkey為null空，則返回redis存儲的前十最熱詞條)
????@Override
????public?List?getHotList(String?searchkey)?{
????????String?key?=?searchkey;
????????Long?now?=?System.currentTimeMillis();
????????List?result?=?new?ArrayList<>();
????????ZSetOperations?zSetOperations?=?redisSearchTemplate.opsForZSet();
????????ValueOperations?valueOperations?=?redisSearchTemplate.opsForValue();
????????Set?value?=?zSetOperations.reverseRangeByScore("title",?0,?Double.MAX_VALUE);
????????//key不為空的時候?推薦相關(guān)的最熱前十名
????????if(StringUtils.isNotEmpty(searchkey)){
????????????for?(String?val?:?value)?{
????????????????if?(StringUtils.containsIgnoreCase(val,?key))?{
????????????????????if?(result.size()?>?9)?{//只返回最熱的前十名
????????????????????????break;
????????????????????}
????????????????????Long?time?=?Long.valueOf(valueOperations.get(val));
????????????????????if?((now?-?time)?????????????????????????result.add(val);
????????????????????}?else?{//時間超過一個月沒搜索就把這個詞熱度歸0
????????????????????????zSetOperations.add("title",?val,?0);
????????????????????}
????????????????}
????????????}
????????}else{
????????????for?(String?val?:?value)?{
????????????????if?(result.size()?>?9)?{//只返回最熱的前十名
????????????????????break;
????????????????}
????????????????Long?time?=?Long.valueOf(valueOperations.get(val));
????????????????if?((now?-?time)?????????????????????result.add(val);
????????????????}?else?{//時間超過一個月沒搜索就把這個詞熱度歸0
????????????????????zSetOperations.add("title",?val,?0);
????????????????}
????????????}
????????}
????????return?result;
????}
?
????//每次點擊給相關(guān)詞searchkey熱度?+1
????@Override
????public?int?incrementScore(String?searchkey)?{
????????String?key?=?searchkey;
????????Long?now?=?System.currentTimeMillis();
????????ZSetOperations?zSetOperations?=?redisSearchTemplate.opsForZSet();
????????ValueOperations?valueOperations?=?redisSearchTemplate.opsForValue();
????????zSetOperations.incrementScore("title",?key,?1);
????????valueOperations.getAndSet(key,?String.valueOf(now));
????????return?1;
????}
?
?
}

核心的部分寫完了，剩下的需要你自己將如上方法融入到你自己的代碼中就行了。

代碼實現(xiàn)過濾不雅文字功能

在springboot 里面寫一個配置類加上@Configuration注解，在項目啟動的時候加載一下，代碼如下：

package?com.***.***.interceptor;
?
import?org.springframework.context.annotation.Configuration;
import?org.springframework.core.io.ClassPathResource;
import?java.io.*;
import?java.util.HashMap;
import?java.util.HashSet;
import?java.util.Map;
import?java.util.Set;
?
?
//屏蔽敏感詞初始化
@Configuration
@SuppressWarnings({?"rawtypes",?"unchecked"?})
public?class?SensitiveWordInit?{
????//?字符編碼
????private?String?ENCODING?=?"UTF-8";
????//?初始化敏感字庫
????public?Map?initKeyWord()?throws?IOException?{
????????//?讀取敏感詞庫?,存入Set中
????????Set?wordSet?=?readSensitiveWordFile();
????????//?將敏感詞庫加入到HashMap中//確定有窮自動機DFA
????????return?addSensitiveWordToHashMap(wordSet);
????}
?
????//?讀取敏感詞庫?,存入HashMap中
????private?Set?readSensitiveWordFile()?throws?IOException?{
????Set?wordSet?=?null;
????????ClassPathResource?classPathResource?=?new?ClassPathResource("static/censorword.txt");
????????InputStream?inputStream?=?classPathResource.getInputStream();
????????//敏感詞庫
????????try?{
????????//?讀取文件輸入流
????????????InputStreamReader?read?=?new?InputStreamReader(inputStream,?ENCODING);
????????????//?文件是否是文件?和?是否存在
????????????wordSet?=?new?HashSet();
????????????//?StringBuffer?sb?=?new?StringBuffer();
????????????// BufferedReader是包裝類，先把字符讀到緩存里，到緩存滿了，再讀入內(nèi)存，提高了讀的效率。
????????????BufferedReader?br?=?new?BufferedReader(read);
????????????String?txt?=?null;
????????????//?讀取文件，將文件內(nèi)容放入到set中
????????????while?((txt?=?br.readLine())?!=?null)?{
????????????????wordSet.add(txt);
????????????}
????????????br.close();
????????????//?關(guān)閉文件流
????????????read.close();
????????}?catch?(Exception?e)?{
????????????e.printStackTrace();
????????}
????????return?wordSet;
????}
????//?將HashSet中的敏感詞,存入HashMap中
????private?Map?addSensitiveWordToHashMap(Set?wordSet)?{
????//?初始化敏感詞容器，減少擴容操作
????Map?wordMap?=?new?HashMap(wordSet.size());
????????for?(String?word?:?wordSet)?{
????????????Map?nowMap?=?wordMap;
????????????for?(int?i?=?0;?i?????????????????//?轉(zhuǎn)換成char型
????????????????char?keyChar?=?word.charAt(i);
????????????????//?獲取
????????????????Object?tempMap?=?nowMap.get(keyChar);
????????????????//?如果存在該key，直接賦值
????????????????if?(tempMap?!=?null)?{
????????????????????nowMap?=?(Map)?tempMap;
????????????????}
????????????????//?不存在則，則構(gòu)建一個map，同時將isEnd設(shè)置為0，因為他不是最后一個
????????????????else?{
????????????????????//?設(shè)置標(biāo)志位
????????????????????Map?newMap?=?new?HashMap();
????????????????????newMap.put("isEnd",?"0");
????????????????????//?添加到集合
????????????????????nowMap.put(keyChar,?newMap);
????????????????????nowMap?=?newMap;
????????????????}
????????????????//?最后一個
????????????????if?(i?==?word.length()?-?1)?{
????????????????????nowMap.put("isEnd",?"1");
????????????????}
????????????}
????????}
????????return?wordMap;
????}
}

然后這是工具類代碼 ：

package?com.***.***.interceptor;
?
import?java.io.IOException;
import?java.util.HashSet;
import?java.util.Iterator;
import?java.util.Map;
import?java.util.Set;
?
//敏感詞過濾器：利用DFA算法??進(jìn)行敏感詞過濾
public?class?SensitiveFilter?{
????//敏感詞過濾器：利用DFA算法??進(jìn)行敏感詞過濾
????private?Map?sensitiveWordMap?=?null;
?
????//?最小匹配規(guī)則
????public?static?int?minMatchType?=?1;
?
????//?最大匹配規(guī)則
????public?static?int?maxMatchType?=?2;
?
????//?單例
????private?static?SensitiveFilter?instance?=?null;
?
????//?構(gòu)造函數(shù)，初始化敏感詞庫
????private?SensitiveFilter()?throws?IOException?{
????????sensitiveWordMap?=?new?SensitiveWordInit().initKeyWord();
????}
?
????//?獲取單例
????public?static?SensitiveFilter?getInstance()?throws?IOException?{
????????if?(null?==?instance)?{
????????????instance?=?new?SensitiveFilter();
????????}
????????return?instance;
????}
?
????//?獲取文字中的敏感詞
????public?Set?getSensitiveWord(String?txt,?int?matchType)?{
????????Set?sensitiveWordList?=?new?HashSet();
????????for?(int?i?=?0;?i?????????????//?判斷是否包含敏感字符
????????????int?length?=?CheckSensitiveWord(txt,?i,?matchType);
????????????//?存在,加入list中
????????????if?(length?>?0)?{
????????????????sensitiveWordList.add(txt.substring(i,?i?+?length));
????????????????//?減1的原因，是因為for會自增
????????????????i?=?i?+?length?-?1;
????????????}
????????}
????????return?sensitiveWordList;
????}
????//?替換敏感字字符
????public?String?replaceSensitiveWord(String?txt,?int?matchType,
???????????????????????????????????????String?replaceChar)?{
????????String?resultTxt?=?txt;
????????//?獲取所有的敏感詞
????????Set?set?=?getSensitiveWord(txt,?matchType);
????????Iterator?iterator?=?set.iterator();
????????String?word?=?null;
????????String?replaceString?=?null;
????????while?(iterator.hasNext())?{
????????????word?=?iterator.next();
????????????replaceString?=?getReplaceChars(replaceChar,?word.length());
????????????resultTxt?=?resultTxt.replaceAll(word,?replaceString);
????????}
????????return?resultTxt;
????}
?
????/**
?????*?獲取替換字符串
?????*
?????*?@param?replaceChar
?????*?@param?length
?????*?@return
?????*/
????private?String?getReplaceChars(String?replaceChar,?int?length)?{
????????String?resultReplace?=?replaceChar;
????????for?(int?i?=?1;?i?????????????resultReplace?+=?replaceChar;
????????}
????????return?resultReplace;
????}
?
????/**
?????*?檢查文字中是否包含敏感字符，檢查規(guī)則如下：

?????*?如果存在，則返回敏感詞字符的長度，不存在返回0
?????*?@param?txt
?????*?@param?beginIndex
?????*?@param?matchType
?????*?@return
?????*/
????public?int?CheckSensitiveWord(String?txt,?int?beginIndex,?int?matchType)?{
????????//?敏感詞結(jié)束標(biāo)識位：用于敏感詞只有1位的情況
????????boolean?flag?=?false;
????????//?匹配標(biāo)識數(shù)默認(rèn)為0
????????int?matchFlag?=?0;
????????Map?nowMap?=?sensitiveWordMap;
????????for?(int?i?=?beginIndex;?i?????????????char?word?=?txt.charAt(i);
????????????//?獲取指定key
????????????nowMap?=?(Map)?nowMap.get(word);
????????????//?存在，則判斷是否為最后一個
????????????if?(nowMap?!=?null)?{
????????????????//?找到相應(yīng)key，匹配標(biāo)識+1
????????????????matchFlag++;
????????????????//?如果為最后一個匹配規(guī)則,結(jié)束循環(huán)，返回匹配標(biāo)識數(shù)
????????????????if?("1".equals(nowMap.get("isEnd")))?{
????????????????????//?結(jié)束標(biāo)志位為true
????????????????????flag?=?true;
????????????????????//?最小規(guī)則，直接返回,最大規(guī)則還需繼續(xù)查找
????????????????????if?(SensitiveFilter.minMatchType?==?matchType)?{
????????????????????????break;
????????????????????}
????????????????}
????????????}
????????????//?不存在，直接返回
????????????else?{
????????????????break;
????????????}
????????}
?
????????if?(SensitiveFilter.maxMatchType?==?matchType){
????????????if(matchFlag?2?||?!flag){????????//長度必須大于等于1，為詞
????????????????matchFlag?=?0;
????????????}
????????}
????????if?(SensitiveFilter.minMatchType?==?matchType){
????????????if(matchFlag?2?&&?!flag){????????//長度必須大于等于1，為詞
????????????????matchFlag?=?0;
????????????}
????????}
????????return?matchFlag;
????}
}

在你代碼的controller層直接調(diào)用方法判斷即可：

//非法敏感詞匯判斷
SensitiveFilter?filter?=?SensitiveFilter.getInstance();
int?n?=?filter.CheckSensitiveWord(searchkey,0,1);
if(n?>?0){?//存在非法字符
????logger.info("這個人輸入了非法字符-->?{},不知道他到底要查什么~?userid-->?{}",searchkey,userid);
????return?null;
}

也可將敏感文字替換*等字符 ：

SensitiveFilter?filter?=?SensitiveFilter.getInstance();
String?text?=?"敏感文字";
String?x?=?filter.replaceSensitiveWord(text,?1,?"*");

最后剛才的 SensitiveWordInit.java 里面用到了 censorword.text 文件，放到你項目里面的 resources 目錄下的 static 目錄中，這個文件就是不雅文字大全，也需要您與時俱進(jìn)的更新，項目啟動的時候會加載該文件。

可以自己百度下載這個東西，很多的，而且與時俱進(jìn)~~，我就不貼鏈接了。