Liferay regularly comes up with latest updates and features over their platform. It has been their hallmark to keep the security updated along with adding optimization functionalities to existing modules. While Elasticsearch has been there on the Liferay Enterprise Solution platform since the launch of Liferay DXP.

Let’s take a look at Elasticsearch, its utility and how it helps custom entities in easy searches.

What is Elasticsearch in Liferay?

Since Liferay is an open source project, its search engine is also an open source. Elasticsearch is a highly scalable, full-text search and analytics engine set as a default search engine in the Liferay. It runs on embedded search engine algorithms and supported in production as a clustering in the Liferay DXP.

Elasticsearch Vs SQL Database

It is a JSON document store that has foundations of Lucene Search Engine from Apache that indexes documents as per strict rules. It locates, manages and indexes complex search queries faster compared to SQL database. It performs analytics and reports is back to deal with data for comparison, graphs and drill-down.

So, when there is a requirement to search documents and libraries inside Liferay Portal Development projects. It provides multiple search parameters and therefore its preferable than an SQL database.

We have written this blog to explore how Elasticsearch helps querying and searching inside a Liferay Portal’s custom entity.

  1. Prerequisite :
    Certain basic knowledge of technologies, their workings and their limitations are required for technical users Some basic knowledge of:
  • Elasticsearch
  • Liferay 7.X
  • Kibana Server
  1. Problem definition:
    Users and Managers within a business face many challenges and their major issue is regarding data. They tend to get affected due to slow search results within their custom entities due to loads of data.

     

      1. Solution:
      2. The Elasticsearch mitigates these issue efficiently. Liferay internally uses Elasticsearch for searching purpose. Example global search, all asset searching, user & many more. We will implement Elasticsearch functionality for one custom entity called ‘Employee’. We have created Employee entity in ‘service.xml’ file in service builder. We are not going in detail about how service builder in Liferay works.

Code Snippet:

  <entity local-service="true" name="Employee" uuid="true">
        <column name="employeeId" primary="true" type="long">
        <column name="name" type="String">
        <column name="age" type="long">
        <column name="address" type="String">
        <column name="city" type="String">
        <column name="companyId" type="long">
        <column name="groupId" type="long">
        <column name="emailAddress" type="String">
</column></column></column></column></column></column></column></column></entity>

  1. First, we would need an indexer which can index our entity through Elasticsearch.
  2. In build.gradle file of the service add the following dependency.
    compileOnly group: “javax.servlet”, name: “javax.servlet-api”, version: “3.0.1”
  3. Create new package in the service layer

(For eg – com.liferay.ka.employee.model.search).

Create a new class EmployeeIndexer that extends BaseIndexer abstract class with Employee as a type argument. All the abstract methods in baseindexer class needs to be implemented it in our indexer. Follow the code snippet.

Code Snippet:

public class EmployeeIndexer extends BaseIndexer{}
  • Above the class Add code snippet @Component annotation.

  • Code Snippet:

    @Component(
    immediate = true,
    service = Indexer.class
    )
    

    Add the EmployeeIndexer constructor

    • Sets the default selected field names. These fields are used to retrieve results documents from the search engine.

    • Make the search results permissions-aware at search time, as well as in the index. Without this, a search query returns all matching employee regardless of the user’s permissions on the resource.

    • If we had to add new field that is as the custom field which is not available as keyword in the Field.class. First, we will have to declare it in the constructor with all the other field itself.

    Code Snippet:

    public EmployeeIndexer() {
    setDefaultSelectedFieldNames(
    Field.ASSET_TAG_NAMES, Field.COMPANY_ID, Field.CONTENT,
    Field.ENTRY_CLASS_NAME, Field.ENTRY_CLASS_PK, Field.GROUP_ID,
    Field.MODIFIED_DATE, Field.SCOPE_GROUP_ID, Field.TITLE, Field.UID,"emailAddress");
    //Set to false in case we are not needing permission check
    setPermissionAware(false);
    }
    
  • In ‘getClassName()’ method you have to return your class name.

  • Code Snippet:

    @Override
    public String getClassName() {
    return Employee.class.getName();
    }
    
  • In ‘postProcessSearchQuery()’ method we use to localize our all the search relevant fields that is to be used for the search results it might be the keyword or our custom field like (TITLE,”emailAddress”).

  • Code Snippet:

    @Override
    public void postProcessSearchQuery(BooleanQuery searchQuery, BooleanFilter fullQueryBooleanFilter,SearchContext searchContext) throws Exception {
    addSearchLocalizedTerm(searchQuery, searchContext, Field.TITLE, false);
    addSearchLocalizedTerm(searchQuery, searchContext, "emailAddress", false);
    }
    
  • ‘doDelete()’ method deletes the document corresponding to the Employee object parameter. Call BaseIndexer’s deleteDocument method with the employee’s company ID and employee ID.

  • Code Snippet:

    @Override
    protected void doDelete(Employee employee) throws Exception {
    deleteDocument(employee.getCompanyId(), employee.getEmployeeId());
    }
    
  • Implement ‘doGetDocument()’ method to select the entity’s fields to build a search document that’s indexed by the search engine. The main searchable field for employee is the employee name and email address, which is stored in a employee search document’s title field and if we have any custom field which is not registered in ‘Field.class’ and we have to create our own then we have to implement is using “addtext” and “addKeyword” methods.

  • Code Snippet:

    @Override
    protected Document doGetDocument(Employee employee) throws Exception {
    Document document = getBaseModelDocument(Employee.class.getName(), employee);
    document.addKeyword(Field.COMPANY_ID, employee.getCompanyId());
    document.addKeyword(Field.GROUP_ID, employee.getGroupId());
    document.addKeyword(Field.SCOPE_GROUP_ID, employee.getGroupId());
    document.addKeyword(Field.TITLE,employee.getName());
    document.addText("emailAddress", employee.getEmailAddress());
    document.addKeyword("emailAddress", employee.getEmailAddress());
    return document;
    }
    
  • We need to implement ‘doGetSummary()’ method to return a summary. It is a text-based version of the entity that can be displayed generically. Call BaseIndexer’s createSummary method, then use summary. SetMaxContentLength to set the summary content’s maximum size.

  • Code Snippet:

    @Override
    protected Summary doGetSummary(Document document, Locale locale, String snippet, PortletRequest portletRequest,PortletResponse portletResponse) throws Exception {
    Summary summary = createSummary(document);
    summary.setMaxContentLength(200);
    return summary;
    }
    
  • In ‘doReindex(Employee employee)’ method, which gets called when a user explicitly triggers a reindex from admin panel. The first doReindex method takes a single object argument. Retrieve the associated document with BaseIndexer’s getDocument method, then invoke IndexWriterHelper’s updateDocument method to update(reindex) the document.

  • Code Snippet:

    @Override
    protected void doReindex(Employee employee) throws Exception {
    Document document = getDocument(employee);
    indexWriterHelper.updateDocument(
    getSearchEngineId(), employee.getCompanyId(), document,
    isCommitImmediately());
    }
    
  • The second ‘doReindex(String className, long classPK)’ method takes two arguments: a className string, and a classPK long. In this method, you retrieve the employee corresponding to the primary key by calling EmployeeLocalService’s getEmployee method, passing in the classPK(primary key) parameter. Then pass the employee to the first doReindex method (see below).

  • Code Snippet:

    @Override
    protected void doReindex(String className, long classPK) throws Exception {
    Employee employee = employeeLocalService.getEmployee(classPK);
    doReindex(employee);
    }
    
  • The third ‘doReindex (String[] ids)’ method indexes all entities in the current Liferay Portal instance (companyId). It takes a string array (ids) as an argument. ‘GetterUtil.getLong(ids[0])’ retrieves the first string in the array, casts it to a long, stores it in a companyId variable, and passes it as an argument to the reindexEmployees helper method.

  • Code Snippet:

    @Override
    protected void doReindex(String[] ids) throws Exception {
    long companyId = GetterUtil.getLong(ids[0]);
    reindexEmployees(companyId);
    }
    
  • To reindex employees, provide the helper method reindexEmployees(long companyId). In this method, use an actionable dynamic query helper method to retrieve all the employees in the Liferay Portal instance. Service Builder generated this query method for you when you built the services. Each employee’s document is then retrieved and added to a collection.

  • Code Snippet:

    protected void reindexEmployees(long companyId) throws PortalException {
    final IndexableActionableDynamicQuery indexableActionableDynamicQuery =
    employeeLocalService.getIndexableActionableDynamicQuery();
    indexableActionableDynamicQuery.setCompanyId(companyId);
    indexableActionableDynamicQuery.setPerformActionMethod(
    new ActionableDynamicQuery.PerformActionMethod() {
    @Override
    public void performAction(Employee employee) {
    try {
    Document document = getDocument(employee);
    indexableActionableDynamicQuery.addDocuments(document);
    }
    catch (PortalException pe) {
    pe.printStackTrace();
    }
    }
    });
    indexableActionableDynamicQuery.setSearchEngineId(getSearchEngineId());
    indexableActionableDynamicQuery.performActions();
    }
    
  • Get the log for the employee model and add the necessary service references in the file.

  • Code Snippet:

    private static final Log log = LogFactoryUtil.getLog(EmployeeIndexer.class);
     
    @Reference
    EmployeeLocalService employeeLocalService;
     
    @Reference
    protected IndexWriterHelper indexWriterHelper;
    
  • Whenever a employee database entity is added, updated, or deleted, the search index must be updated accordingly. The Liferay Portal annotations @Indexable and @IndexableType mark your service methods so documents can be updated or deleted. You must update the addEmployee, updateEmployee, and deleteEmployee service methods with these annotations.
    1. Open EmployeeLocalServiceImpl in the employee-service module’s add the following annotation above the method signature for the addEmployee,updateEmployee and deleteEmployee methods.
    Code Snippet:

    public EmployeeSearchBean searchEmployee(SearchContext searchContext,String
    action) {
    int total = 0;
    Hits hits;
    EmployeeSearchBean employeeSearchBean = new EmployeeSearchBean();
    List employeeList = new ArrayList<>();
    EmployeeIndexer indexer = (EmployeeIndexer)
    IndexerRegistryUtil.getIndexer(Employee.class);
    try {
    if (action.equalsIgnoreCase("advanceSearch")) {
    BooleanQuery booleanQuery = getBooleanQuery(searchContext);
    hits = IndexSearcherHelperUtil.search(searchContext, booleanQuery);
    }else{
    hits = indexer.search(searchContext);
    }
    total = hits.getLength();
    for (int i = 0; i < hits.getDocs().length; i++) {
    Document doc = hits.doc(i);
    long employeeId = GetterUtil.getLong(doc.get(Field.ENTRY_CLASS_PK));
    Employee employee = null;
    employee = EmployeeLocalServiceUtil.getEmployee(employeeId);
    employeeList.add(employee);
    }
    } catch (PortalException | SystemException e) {
    e.printStackTrace();
    }
    employeeSearchBean.setEmployeeList(employeeList);
    employeeSearchBean.setTotal(total);
    return employeeSearchBean;
    }
     
    private BooleanQuery getBooleanQuery(SearchContext searchContext) {
    BooleanQuery booleanQuery = new BooleanQueryImpl();
    BooleanQuery fullQuery = (BooleanQuery) booleanQuery.addRequiredTerm(Field.GROUP_ID,
    (long) searchContext.getAttribute("groupId"));
     
    fullQuery.addRequiredTerm(Field.ENTRY_CLASS_NAME, Employee.class.getName());
    if (Validator.isNotNull(searchContext.getAttribute("name"))) {
     
    fullQuery.addRequiredTerm(Field.TITLE, (String) searchContext.getAttribute("name"));
    }
    if (Validator.isNotNull(searchContext.getAttribute("emailAddress"))) {
     
    fullQuery.addRequiredTerm("emailAddress", (String) searchContext.getAttribute("emailAddress"));
    }
    return fullQuery;
    }
    
  • In getSearchContext() method we set keyword passed from jsp and store it in the serializable map and add the required other fields like companyId, scopeGroupId etc in this map and return it to the setSearchListing() method.

  • Code Snippet:

    public EmployeeSearchBean searchEmployee(SearchContext searchContext,String
    action) {
    int total = 0;
    Hits hits;
    EmployeeSearchBean employeeSearchBean = new EmployeeSearchBean();
    List employeeList = new ArrayList<>();
    EmployeeIndexer indexer = (EmployeeIndexer)
    IndexerRegistryUtil.getIndexer(Employee.class);
    try {
    if (action.equalsIgnoreCase("advanceSearch")) {
    BooleanQuery booleanQuery = getBooleanQuery(searchContext);
    hits = IndexSearcherHelperUtil.search(searchContext, booleanQuery);
    }else{
    hits = indexer.search(searchContext);
    }
    total = hits.getLength();
    for (int i = 0; i < hits.getDocs().length; i++) {
    Document doc = hits.doc(i);
    long employeeId = GetterUtil.getLong(doc.get(Field.ENTRY_CLASS_PK));
    Employee employee = null;
    employee = EmployeeLocalServiceUtil.getEmployee(employeeId);
    employeeList.add(employee);
    }
    } catch (PortalException | SystemException e) {
    e.printStackTrace();
    }
    employeeSearchBean.setEmployeeList(employeeList);
    employeeSearchBean.setTotal(total);
    return employeeSearchBean;
    }
     
    private BooleanQuery getBooleanQuery(SearchContext searchContext) {
    BooleanQuery booleanQuery = new BooleanQueryImpl();
    BooleanQuery fullQuery = (BooleanQuery) booleanQuery.addRequiredTerm(Field.GROUP_ID,
    (long) searchContext.getAttribute("groupId"));
     
    fullQuery.addRequiredTerm(Field.ENTRY_CLASS_NAME, Employee.class.getName());
    if (Validator.isNotNull(searchContext.getAttribute("name"))) {
     
    fullQuery.addRequiredTerm(Field.TITLE, (String) searchContext.getAttribute("name"));
    }
    if (Validator.isNotNull(searchContext.getAttribute("emailAddress"))) {
     
    fullQuery.addRequiredTerm("emailAddress", (String) searchContext.getAttribute("emailAddress"));
    }
    return fullQuery;
    }
    
  • In employee-web, open the file view.jsp. Add a render URL near the top of the file, just after the scriptlet.

  • Code Snippet:

    public void setSearchListing(RenderRequest renderRequest, RenderResponse renderResponse,EmployeeLocalService employeeLocalService) {
     
    PortletURL iteratorURL = PortletURLUtil.getCurrent(renderRequest, renderResponse);
    SearchContainer employeeSearchContainer = new SearchContainer<>(renderRequest, iteratorURL, null,StringPool.BLANK);
    String keywords = ParamUtil.getString(renderRequest, EmployeePortletConstant.KEYWORD);
    String action = ParamUtil.getString(renderRequest, EmployeePortletConstant.ACTION);
    SearchContext searchContext = getSearchContext(renderRequest, employeeSearchContainer);
    EmployeeSearchBean employeeSearchBean = employeeLocalService.searchEmployee(searchContext,action);
    employeeSearchContainer.setResults(employeeSearchBean.getEmployeeList());
    employeeSearchContainer.setTotal(employeeSearchBean.getTotal());
    renderRequest.setAttribute(EmployeePortletConstant.EMPLOYEE_SEARCH_CONTAINER, employeeSearchContainer);
    if (EmployeePortletConstant.ADVANCE_SEARCH.equals(action)) {
    renderRequest.setAttribute(EmployeePortletConstant.KEYWORD, action);
    } else {
    renderRequest.setAttribute(EmployeePortletConstant.KEYWORD, keywords);
    }
    }
    
    Search result through custom indexer :
    1. Kibana

    Note : Kibana server version should be same as Elasticsearch or higher than Elasticsearch version

    • First you have to create index in kibana server.To create Index go to Management and then select Index Patterns
    • Index name would be in format “liferay-companyId”.
    Index-Patterns
    • In Time Filter field select “I don’t want to use the Time Filter”and click on “Create Index Pattern
    • After clicking on Discover you will see all the results related to that companyId, you can search custom entity keyword from the search bar and result will be shown.
    companyId

    Blog Written By Harsh Soni, Jr. Consultant, Anblicks

    Cloud Cost Management Logo Web
    Hadoop to Snowflake New Web
    capptixAI-case-study
    lendingAI-logo_opt
    CustomerAI_Logo_700x300 copy
    sales-ai-logo-500x200