An Error Occurred While Executing the Search Please Try Again

May 18, 2022 Post a Comment

Troubleshooting

Problem

A ruby-red bar with the []An IO Error occurred on server(due south) 10.x.ten.ten . Please endeavour again. bulletin is displayed while running searches.

Symptom

When running historical searches, a red bar is displayed on the results page. It displays the message: An IO Error occurred on server(south) hostname . Please endeavour again. The Hostname or IP address that is displayed in the message is probable that of the console appliance. Applying filters to the search by using the "Issue Processor" parameter might eliminate the mistake fifty-fifty when including the console appliance.

Cause

Receiving the An IO Error occurred on server(s) hostname . Delight try again. message while running searches indicates that the Ariel database is not accessible on one or more managed hosts. The Hostname or IP address that is displayed on the error message does not ever match the host or hosts that are experiencing the problem as there are various reasons that cause this error bulletin to be displayed.

Notation: Whether a hostname or an IP address is included in the message tin depend on your proper name resolution configuration. Both possibilities are included in this certificate.

Diagnosing The Problem

When you are running a historical search, the panel volition proxy your search requests to other managed hosts involved depending on your specified filters. An IO Fault indicates that one or more of your managed hosts are non responding to these search requests. Identifying the right host or hosts that are experiencing the effect is the showtime step in effectively troubleshooting this problem. Y'all can use a combination of the methods below to help you identify the managed host that is experiencing the upshot. If you have a minor number of managed hosts or you already suspect a managed host, you can skip to the Checking the Security Information Distribution tab section for the verification step only.

Search Details
By clicking the More Details link at the results section, you can become a better movie of how your managed hosts are responding to your search:

In this case, the host test-ep ran its search on no information files and the search elapsing was too cipher:

Based on this result, test-ep is the host that is experiencing the problem. In a more realistic example, it tin can exist necessary to verify your findings.

Reviewing the QRadar logs
If your managed hosts are not encrypted, or you have a mix of encrypted and not encrypted managed hosts in your environment, QRadar logs can be useful in identifying the managed host having the problems. Run the search by using the following filters to identify the actual mistake bulletin:

Result Processor: your console
QuickFilter: MappingFactory
Time window: A fourth dimension range that includes the last time that you received the IO Error

If the managed host experiencing the Ariel database consequence is not encrypted, the raw result includes its host name. Drill into the outcome and discover the raw text:

Sep 14 fourteen:56:23 127.0.0.one [aqw_remote_2:4bac31ad-5cb4-47c8-8f0d-280d3bcb3d10] com.q1labs.ariel.searches.tasks.ServiceTaskBase: [Mistake] [Non:0000003000][198.51.100.two/- -] [-/- -]Can't communicate to server [ exam-ep:32006 ] executing query:Id:4bac31ad-5cb4-47c8-8f0d-280d3bcb3d10, DB:<events@/store/ariel/events/records, /store/ariel/events/payloads>, Time:<16-09-fourteen,14:55:23 to 16-09-fourteen,xiv:56:00>, Criteria=<DeviceType:[368,368]>, MappingFactory=com.q1labs.core.types.event.mapping.NormalizedEventMappingFactory@4ee, processedRecordsLimit=2147483647, executionTimeLimit=9223372036854775807, collectedRecordsLimit=2147483647, prio=NORMAL

If the event is on an encrypted upshot processor, the raw event will instead contain localhost as the hostname:

Sep 14 14:23:23 127.0.0.1 [aqw_remote_2:dd380d0d-ad31-4497-a9d3-81224cbd4b6b] com.q1labs.ariel.searches.tasks.ServiceTaskBase: [ERROR] [NOT:0000003000][198.51.100.2/- -] [-/- -]Tin't communicate to server [ localhost:32006 ] executing query:Id:dd380d0d-ad31-4497-a9d3-81224cbd4b6b, DB:<events@/shop/ariel/events/records, /store/ariel/events/payloads>, Time:<sixteen-09-14,14:22:23 to 16-09-14,fourteen:23:00>, Criteria=<DeviceType:[368,368]>, MappingFactory=com.q1labs.cadre.types.event.mapping.NormalizedEventMappingFactory@4ee, processedRecordsLimit=2147483647, executionTimeLimit=9223372036854775807, collectedRecordsLimit=2147483647, prio=NORMAL

Notation: Regardless of the encryption setting of your managed host, you lot should brand a note of host proper name information from these raw events, as it is useful when verifying connectivity equally described in the Resolving the Problem section.

Eliminating Event Processors
The IO Error will only be displayed when you are searching on the managed host experiencing the issue. In our example, setting a filter to evidence events only from the panel eliminates the IO errror:

Therefore, you can prepare filters on your search to assistance y'all identify which managed hosts are experiencing the effect. Try filtering on the managed hosts that you previously identified when checking the search details. If the Ariel database of the managed host or hosts that you are filtering on is not accessible, y'all receive the same IO error:

Checking the Security Data Distribution tab
Once yous accept an idea about which managed host or hosts are experiencing an issue, you can verify your conclusion by checking the Security Data Distribution tab of the System Data window. Open this tab by clicking Admin > System Configuration > Organization and License Management > Systems. When Arrangement and License Direction window is opened, select the suspected managed host and click Deportment > View and Manage System. The Organization and License Details window open with Security Data Distribution tab that is selected past default. If the Ariel database is non accessible, the following warning is displayed:

Resolving The Problem

When you accept identified which Issue Processor is experiencing the issue, you lot need to restore the admission to its Ariel database. This is not e'er lilliputian. Below are some basic resolution steps that can help address the well-nigh common causes before reaching out IBM support for farther assistance.

Warning: The deployment of full configuration that is recommended in some of the below steps restart the services on all of your managed hosts, which outcome in a cursory service break. This break must be taken into consideration when deploying total configuration. Perform a Full Deploy by going to the Admin tab on the UI and clicking Advanced > Deploy Full Configuration.

Open an SSH connection to your Panel with the root account.
Create an SSH connection to the managed Host that yous identified in the Diagnosing the Problem Section.
Case:

[root@examination-console ~]# ssh examination-ep

If you are not able to connect to your Managed Host, verify that your host is powered up and your network connectivity is correctly routing to the managed host IP address on port 22. If your host is operational and your network connectivity is verified merely you are still unable to connect to the host, contact back up for further assistance.
Verify that the Ariel Query Server is running on this managed host:
Example:

[root@test-ep ~]# /opt/qradar/init/ariel_query_server condition
ariel (pid 13732) is running...

If the Ariel Query Server is not running, a full configuration deployment may resolve this issue by restarting all services on the managed host after deploying the almost contempo configuration on it. If the Ariel Query Server is still not running later on a full deployment, contact support for further assistance.
If the Ariel Query Server is running, verify that it is listening on the port that is identified in the Diagnosing the Problem section:
Example:
[root@test-ep ~]# netstat -nalp | grep 32006
tcp 0 0 :::32006 :::* LISTEN 13732/ariel

If your Ariel Query Server is running but is not listening on the port identified, a full deployment may resolve the issue past deploying the near contempo configuration on the managed host. If it is still not listening on the port afterward a full deployment, contact support for further assistance.
If your Ariel Query Server is listening on the specified port, verify the connectivity from the panel to the managed host on that specific port. For unencrypted hosts, you demand to use the host proper name or IP address of the managed host and for encrypted host you need to use localhost.
Case for unencrypted hosts:

[root@examination-panel ~]# telnet exam-ep 32006

Instance for encrypted hosts:

[root@exam-console ~]# telnet localhost 32006

If you practice non receive a message indicating a successful connection, the most likely reason is a firewall blocking the traffic for the Ariel port.

Where practice you find more data?

[{"Product":{"lawmaking":"SSBQAC","label":"IBM Security QRadar SIEM"},"Business concern Unit of measurement":{"code":"BU059","characterization":"IBM Software w\/o TPS"},"Component":"Log Action","Platform":[{"code":"PF016","label":"Linux"}],"Version":"Version Independent","Edition":"","Line of Business":{"code":"LOB24","characterization":"Security Software"}}]

johnsonweepty.blogspot.com

Source: https://www.ibm.com/support/pages/qradar-understanding-io-errors-while-searching