Elasticsearch應(yīng)用之京東搜索
點(diǎn)擊上方藍(lán)色字體，選擇“置頂或者星標(biāo)”?
優(yōu)質(zhì)文章第一時(shí)間送達(dá)！
京東搜索Elasticsearch
開發(fā)環(huán)境
elasticsearch 7.10.1 集成IDE idea elasticsearch-head maven 3.6.3
所有開發(fā)環(huán)境 全棧自學(xué)社區(qū) 公眾號(hào)回復(fù) 電腦環(huán)境 關(guān)鍵字即可獲取.
項(xiàng)目概況
pom.xml
<project?xmlns="http://maven.apache.org/POM/4.0.0"?xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
?????????xsi:schemaLocation="http://maven.apache.org/POM/4.0.0?https://maven.apache.org/xsd/maven-4.0.0.xsd">
????<modelVersion>4.0.0modelVersion>
????<parent>
????????<groupId>cn.com.codingcegroupId>
????????<artifactId>codingce-esartifactId>
????????<version>0.0.1-SNAPSHOTversion>
????????<relativePath/>?
????parent>
????<groupId>cn.com.codingcegroupId>
????<artifactId>codingce-es-jdartifactId>
????<version>0.0.1-SNAPSHOTversion>
????<name>codingce-es-jdname>
????<description>Demo?project?for?Spring?Bootdescription>
????<properties>
????????<java.version>1.8java.version>
????????
????????<elasticsearch.version>7.10.1elasticsearch.version>
????properties>
????<dependencies>
????????<dependency>
????????????<groupId>org.springframework.bootgroupId>
????????????<artifactId>spring-boot-starter-data-elasticsearchartifactId>
????????dependency>
????????<dependency>
????????????<groupId>org.springframework.bootgroupId>
????????????<artifactId>spring-boot-starter-thymeleafartifactId>
????????dependency>
????????<dependency>
????????????<groupId>org.springframework.bootgroupId>
????????????<artifactId>spring-boot-starter-webartifactId>
????????dependency>
????????<dependency>
????????????<groupId>org.springframework.bootgroupId>
????????????<artifactId>spring-boot-devtoolsartifactId>
????????????<scope>runtimescope>
????????????<optional>trueoptional>
????????dependency>
????????<dependency>
????????????<groupId>org.springframework.bootgroupId>
????????????<artifactId>spring-boot-configuration-processorartifactId>
????????????<optional>trueoptional>
????????dependency>
????????<dependency>
????????????<groupId>org.projectlombokgroupId>
????????????<artifactId>lombokartifactId>
????????????<optional>trueoptional>
????????dependency>
????????<dependency>
????????????<groupId>org.springframework.bootgroupId>
????????????<artifactId>spring-boot-starter-testartifactId>
????????????<scope>testscope>
????????dependency>
????????
????????<dependency>
????????????<groupId>org.jsoupgroupId>
????????????<artifactId>jsoupartifactId>
????????????<version>1.10.2version>
????????dependency>
????????<dependency>
????????????<groupId>com.alibabagroupId>
????????????<artifactId>fastjsonartifactId>
????????????<version>1.2.73version>
????????????<scope>compilescope>
????????dependency>
????dependencies>
????<build>
????????<plugins>
????????????<plugin>
????????????????<groupId>org.springframework.bootgroupId>
????????????????<artifactId>spring-boot-maven-pluginartifactId>
????????????????<configuration>
????????????????????<excludes>
????????????????????????<exclude>
????????????????????????????<groupId>org.projectlombokgroupId>
????????????????????????????<artifactId>lombokartifactId>
????????????????????????exclude>
????????????????????excludes>
????????????????configuration>
????????????plugin>
????????plugins>
????build>
project>
實(shí)現(xiàn)代碼
ElasticSearchConfig
這里僅僅是單個(gè) Elasticsearch
/**
?*?ElasticSearch配置類
?*?找到對(duì)象,?放到Spring里面就可以用了
?*
?*?@author?mxz
?*/
@Configuration
public?class?ElasticSearchConfig?{
????@Bean
????public?RestHighLevelClient?restHighLevelClient()?{
????????RestHighLevelClient?client?=?new?RestHighLevelClient(
????????????????RestClient.builder(
????????????????????????new?HttpHost("localhost",?9200,?"http")));
//????????????????????????new?HttpHost("localhost",?9201,?"http")));
????????return?client;
????}
}
實(shí)體類 Content
用于對(duì)應(yīng)京東單個(gè)商品數(shù)據(jù)
/**
?*?實(shí)體類
?*
?*?@author?mxz
?*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public?class?Content?{
????private?String?title;
????private?String?img;
????private?String?price;
}
工具類 HtmlParseUtil
用于解析京東搜索的數(shù)據(jù)
/**
?*?@author?mxz
?*/
@Component
public?class?HtmlParseUtil?{
????public?List?parseJD(String?keywords)?throws?Exception? {
????????//?獲取請(qǐng)求?https://search.jd.com/Search?keyword=java
????????String?url?=?"https://search.jd.com/Search?keyword="?+?keywords;
????????//?解析網(wǎng)頁?(返回?Document?就是瀏覽器?Document?對(duì)象)
????????Document?document?=?Jsoup.parse(new?URL(url),?30000);
????????//?所有在js中可以使用的方法,?這里都可以使用
????????Element?element?=?document.getElementById("J_goodsList");
//????????System.out.println(element.html());
????????ArrayList?goodList?=?new?ArrayList<>();
????????//?獲取所有的li元素
????????Elements?elements?=?element.getElementsByTag("li");
????????//?獲取元素中的內(nèi)容
????????for?(Element?el?:?elements)?{
????????????//?關(guān)于這種圖片特別多的網(wǎng)站,?所有的圖片都是延遲加載的
????????????String?img?=?el.getElementsByTag("img").eq(0).attr("data-lazy-img");
????????????String?price?=?el.getElementsByClass("p-price").eq(0).text();
????????????String?title?=?el.getElementsByClass("p-name").eq(0).text();
????????????Content?content?=?new?Content(title,?img,?price);
????????????goodList.add(content);
????????}
????????return?goodList;
????}
????public?static?void?main(String[]?args)?throws?IOException?{
????????//?獲取請(qǐng)求?https://search.jd.com/Search?keyword=java
????????String?url?=?"https://search.jd.com/Search?keyword=java";
????????//?解析網(wǎng)頁?(返回?Document?就是瀏覽器?Document?對(duì)象)
????????Document?document?=?Jsoup.parse(new?URL(url),?30000);
????????//?所有在js中可以使用的方法,?這里都可以使用
????????Element?element?=?document.getElementById("J_goodsList");
//????????System.out.println(element.html());
????????//?獲取所有的li元素
????????Elements?elements?=?element.getElementsByTag("li");
????????//?獲取元素中的內(nèi)容
????????for?(Element?el?:?elements)?{
????????????//?關(guān)于這種圖片特別多的網(wǎng)站,?所有的圖片都是延遲加載的???source-data-Lazy-img
????????????String?img?=?el.getElementsByTag("img").eq(0).attr("src");
????????????String?price?=?el.getElementsByClass("p-price").eq(0).text();
????????????String?title?=?el.getElementsByClass("p-name").eq(0).text();
????????????System.out.println("=====================================");
????????????System.out.println(img);
????????????System.out.println(price);
????????????System.out.println(title);
????????}
????}
}
分析京東搜索商品
業(yè)務(wù)邏輯層 ContentService
/**
?*?業(yè)務(wù)邏輯層
?*
?*?@author?mxz
?*/
@Service
public?class?ContentService?{
????public?static?final?String?ES_INDEX?=?"jd_goods";
????@Autowired
????@Qualifier("restHighLevelClient")
????private?RestHighLevelClient?client;
????/**
?????*?1?解析數(shù)據(jù)?放入?es?中
?????*
?????*?@param?keywords
?????*?@return
?????*?@throws?Exception
?????*/
????public?Boolean?parseContent(String?keywords)?throws?Exception?{
????????List?contents?=?new?HtmlParseUtil().parseJD(keywords);
????????//?查詢出來的數(shù)據(jù)放入到?es?中
????????BulkRequest?bulkRequest?=?new?BulkRequest();
????????bulkRequest.timeout(TimeValue.timeValueSeconds(2));
????????for?(int?i?=?0;?i?
????????????bulkRequest.add(new?IndexRequest(ES_INDEX)
????????????????????.source(JSON.toJSONString(contents.get(i)),?XContentType.JSON));
????????}
????????BulkResponse?bulkResponse?=?client.bulk(bulkRequest,?RequestOptions.DEFAULT);
????????return?!bulkResponse.hasFailures();
????}
????/**
?????*?2?獲取這些數(shù)據(jù)??實(shí)現(xiàn)搜索功能
?????*
?????*?@param?keywords
?????*?@param?pageNo
?????*?@param?pageSize
?????*?@return
?????*?@throws?IOException
?????*/
????public?List>?searchPage(String?keywords,?int?pageNo,?int?pageSize)?throws?IOException?{
????????if?(pageNo?<=?1)?{
????????????pageNo?=?1;
????????}
????????//?條件搜索
????????SearchRequest?searchRequest?=?new?SearchRequest(ES_INDEX);
????????SearchSourceBuilder?searchSourceBuilder?=?new?SearchSourceBuilder();
????????//?分頁
????????searchSourceBuilder.from(pageNo);
????????searchSourceBuilder.size(pageSize);
????????//精準(zhǔn)匹配
????????TermQueryBuilder?termQuery?=?QueryBuilders.termQuery("title",?keywords);
????????searchSourceBuilder.query(termQuery);
????????searchSourceBuilder.timeout(TimeValue.timeValueSeconds(60));
????????//?執(zhí)行搜索
????????searchRequest.source(searchSourceBuilder);
????????//?通過客戶端查詢
????????SearchResponse?searchResponse?=?client.search(searchRequest,?RequestOptions.DEFAULT);
????????//?解析結(jié)果
????????List>?list?=?new?ArrayList<>();
????????for?(SearchHit?documentFields?:?searchResponse.getHits().getHits())?{
????????????list.add(documentFields.getSourceAsMap());
????????}
????????return?list;
????}
????/**
?????*?3?實(shí)現(xiàn)搜索功能高亮
?????*
?????*?@param?keywords
?????*?@param?pageNo
?????*?@param?pageSize
?????*?@return
?????*?@throws?IOException
?????*/
????public?List>?searchPageHighlighter(String?keywords,?int?pageNo,?int?pageSize)?throws?IOException?{
????????if?(pageNo?<=?1)?{
????????????pageNo?=?1;
????????}
????????//?條件搜索
????????SearchRequest?searchRequest?=?new?SearchRequest(ES_INDEX);
????????SearchSourceBuilder?searchSourceBuilder?=?new?SearchSourceBuilder();
????????//?分頁
????????searchSourceBuilder.from(pageNo);
????????searchSourceBuilder.size(pageSize);
????????//精準(zhǔn)匹配
????????TermQueryBuilder?termQuery?=?QueryBuilders.termQuery("title",?keywords);
????????searchSourceBuilder.query(termQuery);
????????searchSourceBuilder.timeout(TimeValue.timeValueSeconds(60));
????????//生成高亮查詢器
????????HighlightBuilder?highlightBuilder?=?new?HighlightBuilder();
????????//高亮查詢字段
????????highlightBuilder.field("title");
????????//如果要多個(gè)字段高亮,這項(xiàng)要為false
????????highlightBuilder.requireFieldMatch(false);
????????//高亮設(shè)置
????????highlightBuilder.preTags("");
????????highlightBuilder.postTags("");
????????//下面這兩項(xiàng),如果你要高亮如文字內(nèi)容等有很多字的字段,必須配置,不然會(huì)導(dǎo)致高亮不全,文章內(nèi)容缺失等
????????//最大高亮分片數(shù)
????????highlightBuilder.fragmentSize(800000);
????????//從第一個(gè)分片獲取高亮片段
????????highlightBuilder.numOfFragments(0);
????????searchSourceBuilder.highlighter(highlightBuilder);
????????//?執(zhí)行搜索
????????searchRequest.source(searchSourceBuilder);
????????//?通過客戶端查詢
????????SearchResponse?searchResponse?=?client.search(searchRequest,?RequestOptions.DEFAULT);
????????//?解析結(jié)果
????????List>?list?=?new?ArrayList<>();
????????for?(SearchHit?documentFields?:?searchResponse.getHits().getHits())?{
????????????//?解析高亮字段
????????????Map?highlightFields?=?documentFields.getHighlightFields();
????????????HighlightField?title?=?highlightFields.get("title");
????????????Map?sourceAsMap?=?documentFields.getSourceAsMap();??//?原來的結(jié)果
????????????//?解析高亮字段?將原來的字段換為高亮字段
????????????//?千萬記得要記得判斷是不是為空,?不然你匹配的第一個(gè)結(jié)果沒有高亮內(nèi)容,?那么就會(huì)報(bào)空指針異常,?這個(gè)錯(cuò)誤一開始真的搞了很久
????????????if?(title?!=?null)?{
????????????????Text[]?fragments?=?title.fragments();
????????????????String?newTitle?=?"";
????????????????for?(Text?fragment?:?fragments)?{
????????????????????newTitle?+=?fragment;
????????????????}
????????????????sourceAsMap.put("title",?newTitle);
????????????????System.out.println(newTitle);
????????????}
????????????list.add(sourceAsMap);
????????}
????????return?list;
????}
}
控制層 RestController
@RestController
public?class?ContentController?{
????@Autowired
????private?ContentService?contentService;
????@GetMapping("/parse/{keywords}")
????public?Boolean?parse(@PathVariable("keywords")?String?keywords)?throws?Exception?{
????????return?contentService.parseContent(keywords);
????}
????@GetMapping("/search/{keywords}/{pageNo}/{pageSize}")
????public?List>?searchPage(@PathVariable("keywords")?String?keywords,
????????????????????????????????????????????????@PathVariable("pageNo")?int?pageNo,
????????????????????????????????????????????????@PathVariable("pageSize")?int?pageSize)?throws?IOException?{
????????//?return?contentService.searchPage(keywords,?pageNo,?pageSize);
????????//?高亮
????????return?contentService.searchPageHighlighter(keywords,?pageNo,?pageSize);
????}
}
前端頁面 myindex.html
html>
<html?lang="zh-en"?xmlns:th="http://www.thymeleaf.org">
<head>
????<meta?charset="UTF-8">
????<title>全棧自學(xué)社區(qū)搜索title>
????<link?rel="stylesheet"?href="https://v4.bootcss.com/docs/4.6/dist/css/bootstrap.min.css">
????<link?rel="apple-touch-icon"?href="https://v4.bootcss.com/docs/4.6/assets/img/favicons/apple-touch-icon.png"
??????????sizes="180x180">
????<link?rel="icon"?href="https://v4.bootcss.com/docs/4.6/assets/img/favicons/favicon-32x32.png"?sizes="32x32"
??????????type="image/png">
????<link?rel="icon"?href="https://v4.bootcss.com/docs/4.6/assets/img/favicons/favicon-16x16.png"?sizes="16x16"
??????????type="image/png">
????<link?rel="mask-icon"?href="https://v4.bootcss.com/docs/4.6/assets/img/favicons/safari-pinned-tab.svg"
??????????color="#563d7c">
????<link?rel="icon"?href="https://v4.bootcss.com/docs/4.6/assets/img/favicons/favicon.ico">
????<meta?name="msapplication-config"?content="/docs/4.6/assets/img/favicons/browserconfig.xml">
????<meta?name="theme-color"?content="#563d7c">
????<style>
????????.bd-placeholder-img?{
????????????font-size:?1.125rem;
????????????text-anchor:?middle;
????????????-webkit-user-select:?none;
????????????-moz-user-select:?none;
????????????-ms-user-select:?none;
????????????user-select:?none;
????????}
????????@media?(min-width:?768px)?{
????????????.bd-placeholder-img-lg?{
????????????????font-size:?3.5rem;
????????????}
????????}
????style>
head>
<body>
<div?id="app">
????<nav?class="navbar?navbar-expand-md?navbar-dark?bg-dark?mb-4">
????????<a?class="navbar-brand"?href="https://v4.bootcss.com/docs/examples/navbar-static/#">全棧自學(xué)社區(qū)a>
????????<button?class="navbar-toggler"?type="button"?data-toggle="collapse"?data-target="#navbarCollapse"
????????????????aria-controls="navbarCollapse"?aria-expanded="false"?aria-label="Toggle?navigation">
????????????<span?class="navbar-toggler-icon">span>
????????button>
????????<div?class="collapse?navbar-collapse"?id="navbarCollapse">
????????????<ul?class="navbar-nav?mr-auto">
????????????????<li?class="nav-item?active">
????????????????????<a?class="nav-link"?href="https://v4.bootcss.com/docs/examples/navbar-static/#">主頁?<span
????????????????????????????class="sr-only">(current)span>a>
????????????????li>
????????????????<li?class="nav-item">
????????????????????<a?class="nav-link"?href="https://v4.bootcss.com/docs/examples/navbar-static/#">友鏈a>
????????????????li>
????????????????<li?class="nav-item">
????????????????????<a?class="nav-link?disabled"?href="https://v4.bootcss.com/docs/examples/navbar-static/#"
???????????????????????tabindex="-1"?aria-disabled="true">控制面板a>
????????????????li>
????????????ul>
????????????<form?class="form-inline?mt-2?mt-md-0">
????????????????<input?class="form-control?mr-sm-2"?v-model="keyword"?type="text"?placeholder="輸入名稱"
???????????????????????aria-label="Search">
????????????????<button?class="btn?btn-outline-success?my-2?my-sm-0"?@click.prevent="searchKey"?type="submit">搜索
????????????????button>
????????????form>
????????div>
????nav>
????<main?role="main"?class="container">
????????<div?class="row">
????????????<div?class="col-md-4"?v-for="result?in?results">
????????????????<img?:src="result.img"?class="img-thumbnail"/>
????????????????<p><a?v-html="result.title">a>p>
????????????????<p?class="lead">{{result.price}}p>
????????????div>
????????div>
????main>
div>
<script?type="text/javascript"?th:src="@{/js/axios.min.js}">script>
<script?type="text/javascript"?th:src="@{/js/vue.min.js}">script>
<script>
????new?Vue({
????????el:?'#app',
????????data:?{
????????????keyword:?'',?//搜索關(guān)鍵字
????????????results:?[]
????????},
????????methods:?{
????????????searchKey()?{
????????????????var?keyword?=?this.keyword;
????????????????console.log(keyword);
????????????????//?對(duì)接后端接口
????????????????axios.get('/search/'?+?keyword?+?'/1/'?+?'10').then(response?=>?{
????????????????????console.log(response);
????????????????????this.results?=?response.data;??//?綁定數(shù)據(jù)
????????????????})
????????????}
????????}
????})
script>
body>
html>
項(xiàng)目運(yùn)行
搜索前
搜索后
文章已上傳gitee https://gitee.com/codingce/hexo-blog
項(xiàng)目地址: https://github.com/xzMhehe/codingce-java
更多推薦內(nèi)容
↓↓↓
如果你喜歡本文
請(qǐng)長(zhǎng)按二維碼，關(guān)注公眾號(hào)
轉(zhuǎn)發(fā)朋友圈，是對(duì)我最大的支持喲
以上，便是今天的分享，希望大家喜歡，覺得內(nèi)容不錯(cuò)的，歡迎「分享」「贊」或者點(diǎn)擊「在看」支持，謝謝各位。
評(píng)論
圖片
表情