CODEWITHSUNDEEP
elasticsearchspring-data
Vishal Shukla
Vishal Shukla

Reputation: 2998

Rolling Index (Dynamic index name) in Spring Data Elasticsearch

I have a use case in which I want to index my document in a separate index based on a certain condition.
For example, I want to store an invoice document to an index with department name suffixed with it.

@Document(indexName="store_{department}", indexStoreType="invoice")
public class InvoiceES{

    // fields
    @Id
    private String id;
    @Field
    private String department;
}

Is it possible to achieve this using Spring Data?

If not, is it planned in a coming release of Spring Data?

Upvotes: 28

Views: 13609

Answers (7)

Krishna Vedula
Krishna Vedula

Reputation: 1791

Based on the pull request link posted earlier, I was able to stitch together a solution. Currently, support is there only for beans of type String, and they can be injected into the attribute using Spring Expression Language (SpEL), with special @ notation.

First Define a Bean that returns the dynamic prefix:

@Bean
String department() {
    return "some-department";
}

Next, use that in the Elastic Document. Please make a note of the special @ notation.

@Document(indexName = "#{@department}_department")

Notes: Actual requirement for Spring.

Upvotes: 0

Roc King
Roc King

Reputation: 481

Spring will create Elasticsearch[Rest]Template bean which implements ElasticsearchOperations.

The ElasticsearchOperations interface offer index(IndexQuery query) function to save a document.

You can use IndexQueryBuilder.withIndexName to overwrite the index name.

Upvotes: 0

Pavel Krutikhin
Pavel Krutikhin

Reputation: 53

I had to solve similar problem to store data with dynamic index name, consisting of current date: "Index-name-YYYY.MM.DD".

I made a component which has getter of formatted date string:

@Component
public class ElasticIndex {

    private static final DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy.MM.dd");

    public String getIndexDate() {
        return formatter.format(LocalDate.now());
    }
}

Thus, your document class will be like that:

@Document(indexName = "my_data-#{elasticIndex.getIndexDate()}", type = "DataDoc")
public class DataDoc {

    @Id
    private String id;
    private String dataToStore;
    ....
}

So each time you save/delete/etc new document to your elasticsearch repository, it gets index name from getIndexDate method of ElasticIndex bean.

Works on Spring Boot 2.2.5, Spring Data Elasticsearch 3.2.5

Upvotes: 3

Pradeep
Pradeep

Reputation: 149

@Document(indexName = "#{'${elasticsearch.index.name}'}", type = "category", shards = 1, replicas = 0, refreshInterval = "-1")

It created index: "${elasticsearch.index.name}"

I tried with spring-data-elasticsearch version 1.1.2.RELEASE and 1.2.0.M1 but according to jira (https://jira.spring.io/browse/DATAES-93) it is fixed in version 1.1 RC1

Upvotes: 1

Tony
Tony

Reputation: 6148

As far as the spring-boot-starter-data-elasticsearch-1.5, you can achieve it by spring el expression:

@Bean
Department department() {
    return new Department();
}

@Document(indexName="store_#{department.name()}", indexStoreType="invoice")
public class InvoiceES{}

You can change the bean's property to change the index you want to save/search:

    invoiceRepo.save(new Invoice(...));
    department.setName("newName");
    invoiceRepo.save(new Invoice(...));

What should be noticed is not to share this bean in multiple thread, which may mess up your index.

Upvotes: 7

jpereira
jpereira

Reputation: 688

The only way I found to do this is doing it manually without @Document anotation:

        client = new TransportClient().addTransportAddress(new InetSocketTransportAddress(nodeId, port));

        IndexResponse response = client.prepareIndex(your_index, type, subid)
                .setSource(jsonBuilder()
                                .startObject()
                                .field("field1", field1))
                                .field("fileld2", field2)
                                ).endObject())
                .execute().actionGet();

Upvotes: -4

Mohsin Husen
Mohsin Husen

Reputation: 1008

Vishal,

Currently spring data elasticsearch does not support this feature. we already have feature request(pull request) which will be added soon with next release.

Have a look on this pull request, https://github.com/spring-projects/spring-data-elasticsearch/pull/56

Upvotes: 0

Related Questions