How to fix search returning hits for missing data in Liferay?

Question

In Liferay we are looking for articles that satisfy certain conditions by using the following code:

Hits hits = indexSearcherHelper.search(searchContext, query);

Search query which we use is defined as:

BooleanFilter filter = new BooleanFilter();
filter.addRequiredTerm(Field.GROUP_ID, globalSiteId);
filter.addRequiredTerm(Field.STATUS, WorkflowConstants.STATUS_APPROVED);
filter.addRequiredTerm("ddmStructureKey", "TEST");
filter.addRequiredTerm("head", true);
MatchAllQuery query = new MatchAllQuery();
query.setPreBooleanFilter(filter);

and this search finds multiple hits. Then we attempt to get the article like this:

JournalArticleResource journalArticleResource = journalArticleResourceLocalService.getArticleResource(GetterUtil.getLong(hits.toList().get(0).get(Field.ENTRY_CLASS_PK)));
JournalArticle article = journalArticleLocalService.getArticle(journalArticleResource.getGroupId(), journalArticleResource.getArticleId());

However, this produces following error:

No JournalArticleResource exists with the primary key 809477.

In 95% of cases, this code works as expected. But in some cases (on some environments), it appears that index search found results which are not valid. Why does this happen?

Can it be that index has some stale records that are from old, already deleted articles? Do we need to reindex the database?

UPDATE 1: I have observed a very strange behaviour of the index search:

The following code:

        for (int counter = 0; counter < 10; counter++)
        {
            System.out.println(counter);
System.out.println(indexSearcherHelper.search(searchContext, query).toList().size());
        }

produces this result:

0
0
1
4
2
7
3
0
4
4
5
7
6
0
7
4
8
7
9
0

There is only 1 result in reality that needs to be found. On all other environments this code keeps finding just one result in all 10 searches, since we added only 1 article.

In this case, however, it keeps finding no results, 4 results, 7 results and keeps repeating the same pattern.

What is going on here? Is database corrupted? Is it Liferay bug? How can the same search return different number of results? (By the way, last year we did a live database migration from one server to another, that is, migration of the database while Liferay was up and running [not too good idea] to reduce the production downtime, so I am afraid that we might be experiencing the database corruption here.)

UPDATE 2: as requested in the comments, here is the version of the Liferay that we are using and an example of the search with values of some fields modified since this is a production example from closed source application.

Version:

• Liferay Community Edition Portal 7.0.4 GA5 (Wilberforce / Build 7004 / October 23, 2017)

System.out.println(hits.toList().get(0));

{
    ddmTemplateKey=[673861],
    entryClassPK=[809477],
    ddm__keyword__673858__LActive_hr_HR=[true],
    publishDate=[20211116063000],
    ddm__keyword__673858__SActive_hr_HR=[false],
    ddm__keyword__673858__GNA_en_US_String_sortable=[ne],
    ddm__text__673858__OList_hr_HR_String_sortable=[32554651079], 
    classNameId=[0],
    ddm__keyword__673858__SActive_en_US_String_sortable=[false],
    ddm__keyword__673858__O_hr_HR_String_sortable=[opis pop upa],
    modified_sortable=[1637050218921],
    title_hr_HR=[Test ss n],
    ddm__keyword__673858__O_en_US=[Opis pop upa],
    version=[2.4],
    ddm__keyword__673858__B_en_US=[grey],
    ddm__keyword__673858__SActive_hr_HR_String_sortable=[false],
    ddm__keyword__673858__OAll_en_US_String_sortable=[false],
    status=[0],
    ddm__keyword__673858__GPA_en_US=[OK],
    publishDate_sortable=[1637044200000],
    content_hr_HR=[OK 32554651079 NE true Opis pop upa all true  Test pop najnoviji Utorak grey false all false /ervices],
    ddm__keyword__673858__TR_en_US=[all],
    ddm__keyword__673858__B_hr_HR=[grey],
    uid=[com.liferay.journal.model.JournalArticle_PORTLET_811280],
    localized_title_en_US_sortable=[test ss n],
    layoutUuid=[],
    ddm__text__673858__OList_en_US=[32554651079],
    ddm__keyword__673858__GNA_hr_HR=[NE],
    ddm__keyword__673858__TR_en_US_String_sortable=[all],
    ddm__keyword__673858__GNA_hr_HR_String_sortable=[ne],
    createDate=[20211115132217],
    ddm__keyword__673858__OAll_hr_HR_String_sortable=[false],
    displayDate_sortable=[1637044200000],
    ddm__keyword__673858__O_en_US_String_sortable=[opis pop upa],
    entryClassName=[com.liferay.journal.model.JournalArticle],
    ddm__keyword__673858__N_en_US=[Test pop najnoviji Utorak],
    ddm__keyword__673858__S_hr_HR_String_sortable=[all],
    userId=[30588],
    localized_title_en_US=[test ss n],
    ddm__keyword__673858__N_hr_HR_String_sortable=[test pop najnoviji utorak],
    ddm__keyword__673858__OListActive_hr_HR=[true],
    ddm__keyword__673858__GPA_hr_HR_String_sortable [ok],
    treePath=[, 673853],
    ddm__keyword__673858__B_en_US_String_sortable=[grey],
    ddm__keyword__673858__S_hr_HR=[all], groupId=[20152],
    ddm__keyword__673858__B_hr_HR_String_sortable=[grey],
    createDate_sortable=[1636982537964],
    classPK=[0],
    ddm__keyword__673858__S_en_US_String_sortable=[all],
    ddm__keyword__673858__GPA_hr_HR=[OK],
    scopeGroupId=[20152],
    articleId_String_sortable=[809475],
    ddm__keyword__673858__OAll_hr_HR=[false],
    modified=[20211116081018],
    ddm__keyword__673858__LActive_hr_HR_String_sortable=[true],
    ddm__keyword__673858__L_hr_HR=[/ervices],
    localized_title_hr_HR_sortable=[test ss n],
    ddm__keyword__673858__L_en_US=[/ervices],
    visible=[true],
    ddmStructureKey=[TEST],
    ddm__keyword__673858__OAll_en_US=[false],
    defaultLanguageId=[hr_HR],
    ddm__keyword__673858__L_hr_HR_String_sortable=[/ervices],
    viewCount_sortable=[0],
    folderId=[673853],
    classTypeId=[673858],
    ddm__text__673858__OList_hr_HR=[32554651079],
    ddm__keyword__673858__TR_hr_HR_String_sortable=[all],
    companyId=[20116],
    rootEntryClassPK=[809477],
    ddm__keyword__673858__LA_en_US_String_sortable=[true],
    displayDate=[20211116063000],
    ddm__keyword__673858__OListActive_hr_HR_String_sortable=[true],
    ddm__keyword__673858__SActive_en_US=[false],
    ddm__keyword__673858__OListActive_en_US=[true],
    ddm__keyword__673858__LActive_en_US=[true],
    content=[OK 32554651079 NE true Opis pop upa all true  Test pop najnoviji Utorak grey false all false /ervices],
    head=[true],
    ddm__keyword__673858__GPA_en_US_String_sortable=[ok],
    ddm__keyword__673858__OListActive_en_US_String_sortable=[true],
    ratings=[0.0],
    expirationDate_sortable=[9223372036854775807],
    viewCount=[0],
    ddm__text__673858__OList_en_US_String_sortable=[32554651079],
    localized_title_hr_HR=[test ss n],
    expirationDate=[99950812133000], 
    ddm__keyword__673858__N_en_US_String_sortable=[test pop najnoviji utorak],
    roleId=[20123, 20124, 20126],
    ddm__keyword__673858__S_en_US=[all],
    articleId=[809475],
    ddm__keyword__673858__N_hr_HR=[Test pop najnoviji Utorak],
    userName=[tuser%40admin -],
    localized_title=[test ss n],
    stagingGroup=[false],
    headListable=[true],
    ddm__keyword__673858__L_en_US_String_sortable=[/ervices],
    ddm__keyword__673858__O_hr_HR=[Opis pop upa],
    ddm__keyword__673858__TR_hr_HR=[all],
    ddm__keyword__673858__GNA_en_US=[NE]
}

Answer 1

Perhaps you are hitting some inconsistencies in your Elasticsearch index: JournalArticles that don't exist in the Database but they exist in the Elasticsearch.

You can double check this and correct it using my Liferay Index Checker, see https://github.com/jorgediaz-lr/index-checker#readme

Once you have installed it, you have to:

1. Check the "Display orphan index entries" option
2. Click on "Check Index"
3. If you have any orphan results, you can remove them clicking on the "Remove orphans" button.

Answer 2

You might be using the wrong service, try using the journalArticleLocalService.

The id of the journal article resource is the id of the journal article plus 1, so if you have more than one article, in most cases it wont produce the error, but will return the wrong article.