OpenSearch Demo

In this lesson, we guide you through working with Amazon OpenSearch by creating and configuring an OpenSearch domain. A domain represents an OpenSearch cluster with specific settings, instance types, instance counts, and storage resources. In this demo, our domain is named "demo."

Domain Creation: Easy Create vs. Standard Create

When creating a domain, you have two options:

• Easy create: Quickly set up the domain with recommended default configurations.
• Standard create: Customize each setting in detail.

For this lesson, we select Standard create to explore the available configuration options.

Domain Configuration Options

Template Selection and Availability Zones

Begin by selecting an environment template:

• For production data, choose production.
• For demos or testing, select dev/test.

You can deploy the domain with or without a dedicated standby node. In this demo, we proceed without a standby node. Next, choose the number of Availability Zones. We select one Availability Zone—although for production deployments, using multiple zones is recommended for high availability.

OpenSearch Version and Instance Configuration

Choose the OpenSearch version you require. You may also select an Elasticsearch version if needed. For this demo, we opt for version 2.9 (the latest).

Next, select the EC2 instance type to run your cluster. Since this is a demo, we choose a cost-effective T3 small instance. Determine the number of nodes per Availability Zone; for example, one node per zone yields a single node. (If you configured three Availability Zones with one node each, you would have three total nodes.)

Storage Options

Configure your storage preferences:

• Choose EBS as the storage type.
• Select the appropriate volume type.
• Define the storage size per node. For this demo, we use a minimal configuration of 10 GB per node.

Optionally, you can enable a dedicated master node; however, for this demonstration, that option remains disabled.

Network and Access Control Settings

Scroll down to configure network settings. For this demo, choose public access. In a production environment, you may wish to deploy within a specific VPC, subnet, and defined security groups.

For access control, you have two options: use IAM or create a master user. In this example, we create a master user by entering a default master username and password that meet security requirements.

Additional options include enabling SAM authentication or Amazon Cognito authentication. Under access policies, this demo uses fine-grained access control without further modifications.

Leave the remaining settings (such as automatic software updates) as default and deploy your cluster.

Accessing the OpenSearch Cluster

After deployment, click the endpoint to connect to your cluster. You will see two URLs: one for sending requests to add or retrieve data and another for accessing the OpenSearch Dashboard. Click the dashboard URL and log in using your master username (for example, "admin") and the password you configured.

During the initial login, a pop-up message may prompt you to choose between specific tenants or a global dashboard. For simplicity, select the global option.

At this point, the cluster will be empty. OpenSearch provides a demo dataset to help you practice with sample data.

Adding Sample Data

When adding data, you will see three sample datasets. For this lesson, select the e-commerce orders dataset. After installation, click "view data" to explore the new configurations created automatically within OpenSearch.

Within the OpenSearch Dashboard, navigate to Management > Index Management > Indices to view the indices created for the dataset. By selecting the specific e-commerce dataset, you can review the index mappings and see which fields are indexed.

Switch to the Discover tab in the OpenSearch Dashboard to view all data points. By default, the date filter covers the last seven days, but you can adjust this range as needed.

Filters allow you to refine the data further. For instance, selecting the "manufacturer" field displays the top five manufacturers. You can also add filters for fields like "day of week" (e.g., filtering for Saturday) to combine multiple criteria.

Uploading Data to OpenSearch

For those interested in uploading their own data and performing queries on an OpenSearch cluster, refer to the AWS Documentation for a demo with commands to upload data and run queries.

Single Document Upload

Use this curl command to upload a single document to the "movies" index. Remember to update the username, password, and domain endpoint as needed.

curl -XPUT -u 'admin:Password123!' 'domain-endpoint/movies/_doc/1' -d '{ "director": "Burton, Tim", "genre": ["Comedy", "Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson", "Pierce Brosnan", "Sarah Jessica Parker"], "title": "Mars Attacks!" }' -H 'Content-Type: application/json'

Bulk Document Upload

For bulk uploads, prepare a file (e.g., bulk_movies.json) with the following content:

{ "index": { "_index": "movies", "_id": "2" } }
{ "director": "Frankenheimer, John", "genre": ["Drama", "Mystery", "Thriller", "Crime"], "year": 1962, "actor": ["Lansbury, Angela", "Sinatra, Frank"] }
{ "index": { "_index": "movies", "_id": "3" } }
{ "director": "Baird, Stuart", "genre": ["Action", "Crime", "Thriller"], "year": 1998, "actor": ["Downey Jr., Robert", "Jones, Tommy Lee", "Snipes, Wesley"] }
{ "index": { "_index": "movies", "_id": "4" } }
{ "director": "Ray, Nicholas", "genre": ["Drama", "Romance"], "year": 1955, "actor": ["Hopper, Dennis", "Wood, Natalie", "Dean, James", "Mineo, Sal"] }

Then execute the following command to perform the bulk upload:

curl -XPOST -u 'admin:Password123!' 'https://search-demo-kt2xqr2r2yoeyqvf7hqjzna2m.us-east-1.es.amazonaws.com/_bulk' --data-binary @bulk_movies.json -H 'Content-Type: application/json'

If successful, your response will indicate that documents have been added (status code 201). An example response might look like this:

{
  "took": 61,
  "errors": false,
  "items": [
    {
      "index": {
        "_index": "movies",
        "_id": "2",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "movies",
        "_id": "4",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}

After uploading, verify the data in the Index Management section of the OpenSearch Dashboard by reviewing the "movies" index mappings. This view will display fields such as actor, director, genre, title, and year.

Searching Data via Dashboards

To search for data in OpenSearch Dashboards:

1. Navigate to Dashboards > Index Patterns and create an index pattern for "movies."
2. Go to the Discover tab.
3. Select the "movies" index and run queries. For example, search for movies featuring a specific actor like Robert Downey Jr.

Deleting the OpenSearch Domain

To conclude this lesson, delete your OpenSearch cluster by following these steps:

1. In the AWS console, navigate to Domains.
2. Select your "demo" domain.
3. Click delete and confirm by typing the domain name ("demo") into the confirmation dialog.