介紹
記錄將elasticsearch集成到spring boot的過程，以及一些簡單的應用和helper類使用。
接入方式
使用spring-boot中的spring-data-elasticsearch,可以使用兩種內置客戶端接入
1、節點客戶端(node client)：
配置文件中設置為local:false,節點客戶端以無數據節點(node-master或node-client)身份加入集群，換言之，它自己不存儲任何數據，但是它知道數據在集群中的具體位置，并且能夠直接轉發請求到對應的節點上。
2、傳輸客戶端(Transport client)：
配置文件中設置為local:true,這個更輕量的傳輸客戶端能夠發送請求到遠程集群。它自己不加入集群，只是簡單轉發請求給集群中的節點。
兩個Java客戶端都通過9300端口與集群交互，使用Elasticsearch傳輸協議(Elasticsearch Transport Protocol)。集群中的節點之間也通過9300端口進行通信。如果此端口未開放，你的節點將不能組成集群。
環境
版本兼容
請一定注意版本兼容問題。這關系到很多maven依賴。Spring Data Elasticsearch Spring Boot version matrix
搭建環境
Spring boot: 1.4.1.RELEASE
spring-data-elasticsearch: 用了最基礎的spring-boot-starter-data-elasticsearch，選擇高版本時需要對于提高es服務版本
elasticsearch: 2.3.0
Maven依賴
|
1
2
3
4
5
6
7
8
9
10
|
<parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>1.4.1.RELEASE</version> <relativePath/> <!-- lookup parent from repository --></parent><dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> |
配置文件
bootstrap.yml
|
1
2
3
4
5
6
7
8
9
10
11
12
|
spring: data: elasticsearch: # 集群名 cluster-name: syncwt-es # 連接節點,注意在集群中通信都是9300端口，否則會報錯無法連接上！ cluster-nodes: localhost:9300,119.29.38.169:9300 # 是否本地連接 local: false repositories: # 倉庫中數據存儲 enabled: true |
調試
啟動
啟動項目，日志出現以下說明代表成功。并且沒有報錯。
復制代碼 代碼如下:
2017-03-30 19:35:23.078 INFO 20881 --- [ main] o.s.d.e.c.TransportClientFactoryBean : adding transport node : localhost:9300
知識點
在Elasticsearch中，文檔歸屬于一種類型(type),而這些類型存在于索引(index)中，我們可以畫一些簡單的對比圖來類比傳統關系型數據庫：
Elasticsearch集群可以包含多個索引(indices)（數據庫），每一個索引可以包含多個類型(types)（表），每一個類型包含多個文檔(documents)（行），然后每個文檔包含多個字段(Fields)（列）
|
1
2
|
Relational DB -> Databases -> Tables -> Rows -> ColumnsElasticsearch -> Indices -> Types -> Documents -> Fields |
Demo
Customer.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
|
/* * Copyright 2012-2013 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */package com.syncwt.www.common.es;import org.springframework.data.annotation.Id;import org.springframework.data.elasticsearch.annotations.Document;@Document(indexName = "es-customer", type = "customer", shards = 2, replicas = 1, refreshInterval = "-1")public class Customer { @Id private String id; private String firstName; private String lastName; public Customer() { } public Customer(String firstName, String lastName) { this.firstName = firstName; this.lastName = lastName; } public String getId() { return this.id; } public void setId(String id) { this.id = id; } public String getFirstName() { return this.firstName; } public void setFirstName(String firstName) { this.firstName = firstName; } public String getLastName() { return this.lastName; } public void setLastName(String lastName) { this.lastName = lastName; } @Override public String toString() { return String.format("Customer[id=%s, firstName='%s', lastName='%s']", this.id, this.firstName, this.lastName); }} |
CustomerRepository.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
/* * Copyright 2012-2013 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */package com.syncwt.www.common.es;import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;import java.util.List;public interface CustomerRepository extends ElasticsearchRepository<Customer, String> { public List<Customer> findByFirstName(String firstName); public List<Customer> findByLastName(String lastName);} |
CustomerController.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
package com.syncwt.www.web;import com.syncwt.www.response.Message;import com.syncwt.www.service.CustomerService;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestMethod;import org.springframework.web.bind.annotation.RestController;import java.io.IOException;/** * @version v0.0.1 * @Description CustomerController * @Creation Date 2017年03月30日 下午8:21 * @ModificationHistory Who When What * -------- ---------- ----------------------------------- */@RestControllerpublic class CustomerController { @Autowired private CustomerService customerService; @RequestMapping(value = "/test", method = RequestMethod.GET) public Message test() throws IOException { customerService.saveCustomers(); customerService.fetchAllCustomers(); customerService.fetchIndividualCustomers(); return Message.SUCCESS; }} |
CustomerService.java
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
|
package com.syncwt.www.service;import com.syncwt.www.common.es.Customer;import com.syncwt.www.common.es.CustomerRepository;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service;import java.io.IOException;/** * @version v0.0.1 * @Description 業務層 * @Creation Date 2017年03月30日 下午8:19 * @ModificationHistory Who When What * -------- ---------- ----------------------------------- */@Servicepublic class CustomerService { @Autowired private CustomerRepository repository; public void saveCustomers() throws IOException { repository.save(new Customer("Alice", "Smith")); repository.save(new Customer("Bob", "Smith")); } public void fetchAllCustomers() throws IOException { System.out.println("Customers found with findAll():"); System.out.println("-------------------------------"); for (Customer customer : repository.findAll()) { System.out.println(customer); } } public void fetchIndividualCustomers() { System.out.println("Customer found with findByFirstName('Alice'):"); System.out.println("--------------------------------"); System.out.println(repository.findByFirstName("Alice")); System.out.println("Customers found with findByLastName('Smith'):"); System.out.println("--------------------------------"); for (Customer customer : repository.findByLastName("Smith")) { System.out.println(customer); } }} |
spring對es的操作方法
spring-data-elasticsearch查詢方法的封裝
1、封裝數據庫基本CRUD(創建（Create）、更新（Update）、讀取（Retrieve）和刪除（Delete）)
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
public interface CrudRepository<T, ID extends Serializable> extends Repository<T, ID> { <S extends T> S save(S entity); T findOne(ID primaryKey); Iterable<T> findAll(); Long count(); void delete(T entity); boolean exists(ID primaryKey); // … more functionality omitted.} |
2、分頁排序查詢
|
1
2
3
4
5
6
7
8
9
10
11
|
public interface PagingAndSortingRepository<T, ID extends Serializable> extends CrudRepository<T, ID> { Iterable<T> findAll(Sort sort); Page<T> findAll(Pageable pageable);}//Accessing the second page by a page size of 20PagingAndSortingRepository<User, Long> repository = // … get access to a beanPage<User> users = repository.findAll(new PageRequest(1, 20)); |
3、計數
|
1
2
3
4
|
public interface UserRepository extends CrudRepository<User, Long> { Long countByLastname(String lastname);} |
4、刪除
|
1
2
3
4
5
6
7
|
public interface UserRepository extends CrudRepository<User, Long> { Long deleteByLastname(String lastname); List<User> removeByLastname(String lastname);} |
5、自定義查詢方法自動注入
聲明一個接口繼承Repository<T, ID>
|
1
|
interface PersonRepository extends Repository<Person, Long> { … } |
接口中自定義方法，在方法名中包含T中字段名
查詢關鍵字包括find…By, read…By, query…By, count…By, and get…By,熟悉直接可以用And and Or連接
|
1
2
3
|
interface PersonRepository extends Repository<Person, Long> {List<Person> findByLastname(String lastname);} |
保證注入了elasticsearch配置
在bootstrap.yml中寫入了spring-data-elasticsearch的配置文件將自動注入
注入調用
|
1
2
3
4
5
6
7
8
9
|
public class SomeClient { @Autowired private PersonRepository repository; public void doSomething() { List<Person> persons = repository.findByLastname("Matthews"); }} |
6、支持Java8 Stream查詢和sql語句查詢
|
1
2
3
4
5
6
7
8
9
10
11
|
@Query("select u from User u")Stream<User> findAllByCustomQueryAndStream();Stream<User> readAllByFirstnameNotNull();@Query("select u from User u")Stream<User> streamAllPaged(Pageable pageable);try (Stream<User> stream = repository.findAllByCustomQueryAndStream()) { stream.forEach(…);} |
7、支持異步查詢
|
1
2
3
4
5
6
7
8
|
@AsyncFuture<User> findByFirstname(String firstname); @AsyncCompletableFuture<User> findOneByFirstname(String firstname);@AsyncListenableFuture<User> findOneByLastname(String lastname); |
支持原生es JavaAPI
1、NativeSearchQueryBuilder構建查詢
|
1
2
3
4
5
6
7
8
9
10
|
@Autowiredprivate ElasticsearchTemplate elasticsearchTemplate;SearchQuery searchQuery = new NativeSearchQueryBuilder() .withQuery(matchAllQuery()) .withFilter(boolFilter().must(termFilter("id", documentId))) .build();Page<SampleEntity> sampleEntities = elasticsearchTemplate.queryForPage(searchQuery,SampleEntity.class); |
2、利用Scan和Scroll進行大結果集查詢
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
SearchQuery searchQuery = new NativeSearchQueryBuilder() .withQuery(matchAllQuery()) .withIndices("test-index") .withTypes("test-type") .withPageable(new PageRequest(0,1)) .build();String scrollId = elasticsearchTemplate.scan(searchQuery,1000,false);List<SampleEntity> sampleEntities = new ArrayList<SampleEntity>();boolean hasRecords = true;while (hasRecords){ Page<SampleEntity> page = elasticsearchTemplate.scroll(scrollId, 5000L , new ResultsMapper<SampleEntity>() { @Override public Page<SampleEntity> mapResults(SearchResponse response) { List<SampleEntity> chunk = new ArrayList<SampleEntity>(); for(SearchHit searchHit : response.getHits()){ if(response.getHits().getHits().length <= 0) { return null; } SampleEntity user = new SampleEntity(); user.setId(searchHit.getId()); user.setMessage((String)searchHit.getSource().get("message")); chunk.add(user); } return new PageImpl<SampleEntity>(chunk); } }); if(page != null) { sampleEntities.addAll(page.getContent()); hasRecords = page.hasNextPage(); } else{ hasRecords = false; } }} |
3、獲取client實例進行節點操作,可以自行封裝Util方法
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
@Autowiredprivate ElasticsearchTemplate elasticsearchTemplate;public void searchHelper() throws IOException { //節點客戶端 // on startup// Node node = nodeBuilder().clusterName("syncwt-es").client(true).node();// Client nodeClient = node.client(); //傳輸客戶端// Settings settings = Settings.settingsBuilder().build();// Client transportClient = TransportClient.builder().settings(settings).build(); Client transportClient = elasticsearchTemplate.getClient(); Customer customer = new Customer("Alice", "Smith"); // instance a json mapper ObjectMapper mapper = new ObjectMapper(); // create once, reuse // generate json String json = mapper.writeValueAsString(customer); System.out.println("--------------------------------jackson mapper"); System.out.println(json); XContentBuilder builder = jsonBuilder() .startObject() .field("firstName", "Alice") .field("latName", "Smith") .endObject(); System.out.println("--------------------------------jsonBuilder"); System.out.println(builder.string()); IndexResponse response = transportClient.prepareIndex("es-customer", "customer") .setSource(jsonBuilder() .startObject() .field("firstName", "Alice") .field("latName", "Smith") .endObject() ) .execute() .actionGet(); System.out.println("--------------------------------response"); System.out.println(response.toString()); // on shutdown// node.close();// nodeClient.close(); transportClient.close(); } |
總結
4、spring-data-elasticsearch對es有很好的支持，但很多高版本在spring-boot中不是很友好。所以，除了spring-boot自動配置的方法，最好掌握代碼動態配置方法。
5、為了操作的便利性,我們往往需要動態索引，因為同一個索引（固定）是無法滿足集群中多業務的。所以后續封裝一個EsUtil類作為基本操作公交類
以上就是本文的全部內容，希望對大家的學習有所幫助，也希望大家多多支持服務器之家。
原文鏈接：http://www.jianshu.com/p/35f9f867f48f?utm_source=tuicool&utm_medium=referral#
