Using OpenSearch in Fedora Linux

OpenSearch is Amazon’s open-source search engine and analytics suite. Individuals, businesses, and organizations can use the service to search for a wide range of information and use visualization tools to better understand user behavior and search trends. This article will discuss how you can use OpenSearch in Fedora Linux.

Prerequisites

• A Linux Distribution (Fedora OS)
• Amazon Web Services (AWS) Account
• OpenSearch and OpenSearch Dashboards

What can OpenSearch do?

OpenSearch provides several features and tools. These are:

• Applications that monitor and debug your cluster.
• Manage security and event information.
• Enable seamless, personalized search results.
• A web-based user interface for searching and browsing search results.
• The ability to search for specific terms or phrases within a document or webpage.
• The ability to filter search results by date, relevance, or other criteria.
• The ability to create and save searches for later use.
• The ability to customize the appearance and functionality of the search results page.
• Advanced analytics and reporting tools to help users understand and analyze search traffic and user behavior.

The following sections will guide you through the basics of creating a domain, uploading test data, and visualizing your information with OpenSearch Dashboards.

What is an OpenSearch Service domain?

An OpenSearch Service domain is a service provided by AWS that allows you to create, manage, and configure your cluster(s) using either the AWS console or the AWS command-line interface (CLI). This tutorial, will use the AWS console to create and configure your domain.

Getting started

To begin the domain setup, launch your preferred browser and log in to your AWS console. Navigate to the Amazon OpenSearch Service page, then click Create domain.

Choose your domain name and leave the Enable custom endpoint box unmarked.

OpenSearch is a fork of Elasticsearch version 7.10. You can choose any version up to Elasticsearch version 7.10 in addition to OpenSearch versions.

Choose Development and testing for your deployment type, the most recent OpenSearch version, and enable compatibility.

Leave Auto-Tune enabled and Add maintenance window unmarked.

The Data nodes options allows you to customize your nodes based on the needs of your applications:

• Availability Zones (AZ)
  • Amazon Web Services (AWS) Availability Zones are physically separate and isolated data center locations within an AWS region. Each Availability Zone is designed to be fault-tolerant, with redundant power, networking, and cooling infrastructure.
• Instance type:
  • Refers to the type of virtual server you’d like to use for your application.
• Number of nodes:
  • The number of nodes you’d like to allocate to each of your AZs.

Since we’re running in a small development setting, set your AZ to 2, your Instance type to t3.small.search, and Number of nodes to 2. Don’t change the default settings for your Storage type, EBS volume type, and EBS storage size per node.

Ignore these options for now, but read on for more information:

• Warm and cold data storage:
  • For use cases that require a cost effective solution for storing large amounts of non-mutable data.
• Dedicated master nodes
  • Allows you to choose how many master nodes you’d like to use for your domain.
• Snapshot configuration:
  • Set to hourly by default.

VPC access is recommended for production environments. You’ll also need to create a master user login to access OpenSearch Dashboards, OpenSearch’s data visualization tool. We’ll discuss how to use OpenSearch dashboards after you configure your domain.

Select Public access and Create master user, and set up your login.

Leave Prepare SAML authentication and Enable Amazon Cognito authentication option boxes unchecked and select Only use fine-grained access control for your access policy.

Select Use AWS owned key, ignore the optional configurations, click Create to create your domain, then wait for your domain to activate.

Using OpenSearch Dashboards

OpenSearch Dashboards is a tool that allows you to create and customize interactive dashboards to visualize the data your site receives from user interaction. These dashboards are visual representations of data from various sources such as logs, metrics, and security events, which can be customized to meet your specific needs, including:

• Dragging and dropping different types of visualizations, such as graphs, maps, and tables, onto a dashboard.
• Filtering and manipulating data to highlight specific trends or patterns.
• Sharing dashboards with other users or embedding them in other applications.
• Collaborating with other users in real-time on the same dashboard.

Navigate to domains and select it from the list.

Click OpenSearch Dashboards URL to access your OpenSearch Dashboard.

You’ll be presented with one of the following screens after you’ve logged into your dashboard:

Visualization options

Click Add sample data to add sample data provided by AWS.

You may select any of the three options. The Sample web logs option will be used, here, to view examples of types of visualization options you can use to analyze your data.

Click Create new to add more visualization options:

Analyze your own data to analyze

You can upload one or more of your documents by entering commands through a CLI.

Add a single document

curl -XPUT -u 'master-user:[master-user-password]' 'domain-endpoint/[domain name]/_doc/1' -d '{"field1": "string1", "field2": ["string3","string4"]}' -H 'Content-Type: application/json'

Add multiple documents

Create a JSON file with your documents and run a command to add multiple documents:

JSON file format:

{ "index" : { "_index": "indexname", "_id" : "2" } }
{"field1": "string1", "field2": ["string2", "string3", "string4"], "field3": 1234, "field4": ["String, 5", "String, 6"]}
{ "index" : { "_index": "indexname", "_id" : "3" } }
{"field5": "string7", "field6": ["string8", "string9", "string10"], "field7": 5678, "field8": ["String, 11", "String, 12"]}
{ "index" : { "_index": "indexname", "_id" : "4" } }
{"field9": "string13", "field10": ["string14", "string15", "string16"], "field11": 1011, "field12": ["String, 17", "String, 18"]}

JSON file naming restrictions:

• All letters must be lowercase.
• Index names cannot begin with _ or – .
• Index names can’t contain spaces, commas, : , ” , * , + , / , \ , | , ? , # , > , or < .

Command to run:

curl -XPOST -u 'master-user:[master-user-password]' 'domain-endpoint/_bulk' --data-binary @bulk_[domain name].json -H 'Content-Type: application/json'

You can now create and configure your own domain and use OpenSearch Dashboards to visualize the data your domain receives.