Cluster Management Tool

About 14 min

Cluster Management Tool

The TimechoDB Cluster Management Tool is a centralized management console for IoTDB/TimechoDB operations teams. It brings server management, cluster deployment, Workbench, Prometheus/Grafana monitoring, file distribution, health checks, and deployment records together in a single web interface.

1.2 Purpose

Unified access: Manage multiple Linux/Unix servers from a browser.
Automated execution: Connect to target hosts over SSH and deploy the database, Workbench, and monitoring components in dependency order.
Traceable operations: Keep deployment records, logs, health check reports, and uploaded files for delivery audits.
Lightweight deployment: Extract the release package and follow the startup instructions below.

1.3 Capabilities

Area	Main Features	Typical Output
Server management	Add and group servers, test connections, and import/export servers in batches	List of accessible servers
Deployment	Deploy IoTDB, Workbench, and monitoring dashboards with one click	Cluster records and instance status
Monitoring	Manage monitoring dashboards and observe cluster health	Monitoring access points
Health checks and auditing	Configure plans and templates, schedule checks, and view report history	Health check reports
Implementation support	Filter deployment records and generate implementation manuals	Delivery documentation
File distribution	Upload and group installation packages, manuals, and scripts	File assets

The main navigation on the left includes Servers, Cluster Management, Workbench Management, Monitoring Dashboard Management, Implementation Manuals, File Upload, and Operations Tools. The expandable Cluster Health Checks menu contains Health Check Configuration and Report History.

The left navigation includes Servers, Cluster Management, Workbench Management, Monitoring Dashboard Management, Implementation Manuals, File Upload, Operations Tools, and the expandable Cluster Health Checks menu.

2 Installation and Deployment

2.1 Obtain the Release Package

Obtain the Cluster Management Tool installation package and upload it to the server.

2.2 Extract the Package and Enter the Installation Directory

tar -zxvf ops-management-platform-*.tar.gz
cd ops-management-platform-*

2.3 Configure the Listen Address or Data Directory

Edit config/runtime.env in the installation directory. You can rename config/runtime.env.example to config/runtime.env before editing it. For example, if the tool is installed under /data/ops-management-platform-*:

cd config/
mv runtime.env.example runtime.env
vim runtime.env

Set the contents of runtime.env as follows. TMPDIR must be an absolute path; choose a directory appropriate for your environment.

APP_ADDR=0.0.0.0:8386
DATABASE_URL=./data/ops-platform.sqlite
UPLOAD_DIR=./data/uploads
TMPDIR=/data/tmp
WEB_DIR=./web
LOG_LEVEL=info
LOG_RETENTION_DAYS=365

2.4 Start and Access the Tool

Run the startup script from the ops-management-platform-* installation directory:

./sbin/start.sh

The following output indicates that startup has completed:

The startup output confirms that the management tool has launched successfully.

Open http://<server-IP>:8386 in a browser. Replace the listen address 0.0.0.0 with the server's actual IP address.

2.5 Log In (Added in v0.3.1)

Opening the URL takes you to the login page. In the current version, the default username and password are both admin.

The login page provides username and password fields; the default credentials are admin and admin.

3 Server Management

Servers are the common entry point for deployment, remote terminals, performance tests, system initialization, and health checks. Add servers and test their connections before performing operations that modify a target host.

3.1 Server List Fields and Status

Column	Meaning	Recommendation
Server Name	Display name in the tool; defaults to the IP address if left blank when creating the server	Use names such as `node-1`, `node-2`, or `environment-role-number`
User	Standard SSH login account, displayed in green	Use for routine connections and non-privileged operations
Group	Server group; the default group is Default Group	Group servers by project, environment, or server room for batch selection
IP / Address	Target host and SSH port, such as `172.20.31.61:22`	Confirm that DNS/IP is reachable from the tool host before changing it
Connectivity	Connection status for each account, including the standard and privileged accounts where applicable	Run Connection Test before deployment
Actions	Three-line menu on the right of each row	Entry point for operations on an individual server

The server list shows server names, users, groups, addresses, connectivity, and a row-level Actions menu.

3.2 Add Servers

3.2.1 Add a Single Server

On the Servers page, click Add Server and fill in the server information. Test the connection before adding the server.

The Add Server dialog collects a server name, group, host and SSH port, account credentials, and a connection test.

Click Add Server to open the dialog.
Enter the name, group, host address, SSH port, and SSH user. The port must be an integer from 1 to 65535.
Enter the username and password.
Click Test Connection. This validates the information currently entered without adding the server to the list.
After the test succeeds, click Add. The tool saves the server information and refreshes the list.

3.2.2 Add Servers in Batches

For a large number of servers, use the import/export feature. Open the import menu in the upper-right corner and download the template.

The import menu provides a downloadable template for preparing a batch of server records.

Open the downloaded file and fill in the configuration.

The spreadsheet template contains the fields used to import server connection details.

Click the server import file selector, upload the completed template, and wait for the import to finish.

3.3 Edit Server Information

Open the target server's Actions menu and click Edit. The form is similar to the Add Server form. Leave the password blank to keep the existing password; credentials are updated only when a new password is entered.

Changing the name or group does not change the target host.
After changing the IP address or SSH port, run Connection Test immediately to verify the new address.
Changing a standard user to root clears the previous privileged account. When changing root to a standard user, configure the privileged account again.
After clicking Save, wait for the list to refresh. If saving fails, retain the original record and correct the fields according to the error message.

The three-line button at the right of each server row opens the Actions menu. It contains Actions, Tests, and Dangerous Operations groups. Check the server name and IP address before performing an operation.

The server Actions menu groups connection, terminal, initialization, installed-service, performance, and deletion operations.

Menu Item	Function
Connection Test	Test the saved standard and privileged account connections and refresh connectivity status
Command-Line Tool	Open the server's remote terminal in the browser
System Initialization	Configure firewall handling and system parameters for one or multiple servers
Installed Services	Check and list registered assets such as JDK, IoTDB, Workbench, monitoring dashboards, GDAL, watchdogs, and Arthas in real time
Edit	Change the server address, accounts, group, and credentials
Performance Test	Collect system information and run lightweight CPU, memory, and disk tests
Performance Test History	List historical tests for the server; click a record for details
Delete (Dangerous Operation)	Delete the server record from the tool; the confirmation dialog states that this cannot be undone

3.4.1 Connection Test

Run a connection test from the Add/Edit Server form or from a saved server's Actions menu. The page displays separate results and messages for the standard and privileged accounts.

3.4.2 Command-Line Tool

Click Command-Line Tool in the Actions menu to open a remote terminal for the target server. Enter commands and view their output in the terminal window.

The browser terminal provides a command input area and displays output from the remote server.

3.4.3 System Initialization

System initialization supports individual servers and batches. Select a firewall policy before starting:

Policy	Behavior	Suitable Scenario
Open Ports	Keep firewalld enabled and open only the ports required for IoTDB, Workbench, monitoring dashboards, and tool access	Recommended default for production environments
Disable Firewall	Stop and disable firewalld	Only for servers where the deployment requirements explicitly call for disabling the firewall

The system initialization dialog presents the firewall handling choices before applying system settings.

Risk warning: Disabling the firewall changes the target host's security boundary. Select this policy only when explicitly required by the project's security plan.

3.4.4 Installed Services and Dependencies

Open Actions → Installed Services. The page checks remote paths and dependency availability in real time and displays registered service assets. Fields include service type/name, status, version, installation path, installation source, most recent reuse, last check time, and actions.

The Installed Services view lists registered software with status, version, path, installation source, and check details.

3.4.5 Performance Test

Performance tests provide routine operational guidance. They are not intended for formal load testing or procurement acceptance. The tool collects basic system information and runs lightweight CPU, memory, and disk tests on the target server. Results are saved in Performance Test History.

Open Actions → Performance Test and verify the target server, connection address, and execution account.
The performance test dialog identifies the target server and lets you review the test before starting it.
Click Start Test. The window contains Information Collection and Performance Test groups. Each item shows its status, value, unit, and reason for failure or being skipped.
Wait for the test to finish. A successful run displays a completion message; a failed run displays an error. A failed individual metric should not hide other results.
The running test view reports collected system information and the status of CPU, memory, and disk checks.

Interpreting results: Lightweight tests support routine operations, trend observation on the same host, and risk identification before deployment. They do not replace formal load tests, capacity assessments, or procurement acceptance tests.

3.4.6 Performance Test History

Click Performance Test History in the Actions menu to view previous tests for the server. The list shows the test time, result, and duration. Click a record to view system information, individual results, and failure reasons.

An empty history is normal until at least one performance test has been run.
Click a record to view the test time, duration, system information, test items, and result summary.

Performance Test History lists previous runs and provides access to each run’s system details and results.

3.4.7 Delete Servers Individually or in Batches

The Delete action is in the Dangerous Operations group. To delete multiple servers, select them in the list. Both individual and batch deletion require confirmation. After confirmation, the deleted server records cannot be recovered in the tool.

The deletion confirmation identifies the server record that will be permanently removed from the tool.

3.5 Search, Groups, Import, and Export

Filter the list by server name, IP address, group, or connectivity status. Select All Groups to display all servers again.

Search by name: Select Server Name and enter all or part of a name.
Search by IP: Select IP Address and enter an address or part of a hostname.
Filter by group: Select Group, then All Groups or a specific group.
Filter by connectivity: Select Status, then all, connected, or failed connections.
Batch import: Download the template, fill in the host, port, username, password, and optional fields, and upload it. Results are classified as created, skipped, or failed, with row numbers and reasons.
Full export: Export the current server assets. Exported files may contain sensitive connection information; store and redact them according to internal security requirements.
Group management: Add, edit, or delete regular groups. Do not casually delete the system default group.

3.6 Existing Deployments (Added in v0.3.1)

Bring an existing cluster under management with one click. If an IoTDB cluster is already deployed and running on the servers, click Existing Deployments. The tool scans and brings the cluster under centralized monitoring and management without reinstalling it.

Click Existing Deployments.
The Existing Deployments action starts discovery of an already running IoTDB cluster.
Add the server hosting a DataNode.
Currently, only DataNodes using port 6667 are supported. Support for other ports is planned for a later version.
The discovery form selects a server hosting a DataNode and specifies the supported service port.
Review the scan results. If scanning fails, check that the cluster has started correctly.
The scan results show the discovered cluster information and indicate whether discovery succeeded.

After the cluster is added, it appears in Cluster Management with its information.

The adopted cluster appears in Cluster Management with its nodes and current status.

4 One-Click Deployment

4.1 How One-Click Deployment Works

One-Click Deployment is available on both the Servers and Cluster Management pages. You can select multiple services. Deployment runs in this order: IoTDB, IoTDB Workbench, and monitoring dashboards.

The One-Click Deployment dialog lets you choose IoTDB, Workbench, and monitoring components.

Select the target servers in the server list and click One-Click Deployment.
Select one or more services: IoTDB, IoTDB Workbench, or monitoring dashboards.
Click Next and confirm the target servers, deployment parameters, and execution order.
Start deployment and monitor task progress. Keep the tool service running during deployment.
After completion, open Cluster Management and verify the cluster records and node distribution.

The deployment confirmation summarizes the selected servers, services, parameters, and execution order.

4.1.1 IoTDB

Deploy IoTDB

Open the IoTDB deployment wizard from the server console and fill in the required information.

The IoTDB deployment wizard collects the initial cluster and server settings.

You can deploy to multiple servers in a batch. Complete the fields and click Next.

The deployment form accepts the parameters used to install IoTDB on one or more selected servers.

Click Next to start installation and deployment. The tool reports success or a failure reason and displays the final deployment results.

The deployment result reports completion status and any failure details for the selected servers.

After deployment, start the cluster.

The cluster controls provide the actions used to start the newly deployed cluster.

View the cluster information.

The cluster details view shows the registered cluster and its node information.

Activate IoTDB

In Cluster Management, if the cluster is not activated, open More Actions → Activate Cluster.

The cluster Actions menu includes the activation command for an inactive cluster.

Copy the displayed machine code and send it to Timecho staff. After receiving the activation code, enter it in the activation code field and click Confirm Activation.

The activation dialog displays the machine code and provides a field for the activation code.

Check the cluster status after activation and confirm that the DataNodes are no longer in ReadOnly status.

The cluster status view confirms the activation state and DataNode status.

4.1.2 Deploy Workbench

On the Servers page, click One-Click Deployment, select IoTDB Workbench, complete the wizard, and click Next.

The Workbench deployment wizard collects settings for the Workbench instance.

The Workbench setup view provides the remaining instance and connection options.

Review the deployment results.

The deployment result shows whether the Workbench instance was installed successfully.

After deployment succeeds, use Workbench Management to view status, start or stop the service, rename instances, synchronize connections, or open Workbench. A typical access URL is http://IP:9190.

4.1.3 Deploy Monitoring Dashboards

On the Servers page, click One-Click Deployment and select Monitoring Dashboard. Follow the wizard to select the Prometheus and Grafana packages, enter their configuration, and upload the corresponding dashboard.

The monitoring deployment wizard selects Prometheus and Grafana packages and their configuration.

The monitoring setup view provides the dashboard upload and deployment options.

Run the precheck and start deployment. After completion, check the Prometheus/Grafana status for each server. You can also start the monitoring dashboard from Monitoring Dashboard Management.

The deployment status view shows the resulting Prometheus and Grafana services.

Prometheus URL: http://IP:9090
Grafana URL: http://IP:3000

4.1.4 Deploy Watchdogs

IoTDB Cluster Watchdog

Open Cluster Management → target cluster → More Actions → Deploy Watchdog.

The cluster Actions menu provides the Deploy Watchdog command.

After the tool identifies the service account, click Deploy.

The watchdog setup identifies the service account that will run the watchdog.

The deployment dialog lets you confirm and start watchdog installation.

Wait for deployment to complete.

The result view reports the watchdog deployment status.

Workbench Watchdog

In Workbench Management, open the instance's Actions menu and select Deploy Watchdog.

The Workbench instance menu provides a Deploy Watchdog action.

Wait for deployment to complete.

Monitoring Dashboard Watchdog

In Monitoring Dashboard Management, open the instance's Actions menu and select Deploy Watchdog.

The monitoring dashboard instance menu provides a Deploy Watchdog action.

Wait for the tool to identify the service account, click Start Deployment, and wait for completion.

5 Cluster Management

5.1 Server Actions

The cluster server actions expose the available operational controls for a node.

5.1.1 View Logs

Open Cluster Management to view ConfigNode and DataNode logs.

The log view lets you select and inspect ConfigNode or DataNode logs.

5.1.2 Edit Configuration Files

In Cluster Management, you can edit the iotdb-system, confignode-env, and datanode-env configuration files.

The configuration editor provides tabs for iotdb-system, confignode-env, and datanode-env.

The editor displays the selected configuration file and provides download and save controls.

5.1.3 Flame Graphs

In Cluster Management, select the flame graph action in the server list.

The server actions include the flame graph collection command.

Select the collection duration and target DataNode.

The flame graph dialog selects a DataNode and the collection duration.

Download the flame graph after generation completes.

5.2 Cluster Operations

5.2.1 View a Cluster

Open Cluster Management.

The Cluster Management page lists the available clusters and their current states.

Open a cluster to view its status and other details.

Opening a cluster displays its node health and cluster details.

5.2.2 CLI Operations

Click the CLI icon in Cluster Management to open the database CLI. Use it to create, read, update, or delete data.

The cluster CLI control opens a browser-based database command line.

5.2.3 More Actions

Open More Actions in Cluster Management to view the additional cluster operations. Select the operation required for your task.

The More Actions menu exposes the additional cluster administration operations.

6 Workbench and Monitoring Dashboard Management

Open Workbench Management and add the cluster to be monitored.

Workbench Management provides the form for adding a cluster connection.

The Workbench list shows the configured cluster connection and its management controls.

Use the same procedure in Monitoring Dashboard Management.

7 File Upload and Group Management

File Upload provides centralized storage for installation packages, configuration templates, health check templates, implementation manuals, and scripts.

Open File Upload and click Add File.
Select the file and check its type and size.
Select a group and submit the upload.
Use Group Management to maintain groups by project, environment, or version.

7.1 Upload, Distribute, and Delete Files

Click Add File, then drag files into the upload area or use the file selector. Files initially appear as pending upload.
The upload area accepts files by drag-and-drop or through the file picker.
Select a file group and submit. Each file displays its upload percentage, completion status, or failure status. If an upload fails, check the information and retry.
The upload list reports per-file progress and whether each upload succeeded or failed.
Select files and click Distribute. Choose accessible servers and a remote directory, such as the default /data/timecho/. The distribution window shows progress for each server. The distributed files can then be selected automatically during One-Click Deployment on the Servers page.
The distribution dialog selects files, destination servers, and a remote directory, and reports progress by server.
After distribution, the uploaded package is available to select in the server deployment workflow.
Change file groups individually or in batches. You can use the system default group, but should not delete it.
Deleting a file removes both its record and the uploaded file and cannot be undone. If batch deletion partially fails, the page displays the numbers of successful and failed deletions.

8 Operations Tools

Cluster operations and the Operations Tools page support flame graphs, thread stacks, and memory or thread dumps. These diagnostic operations usually create temporary files or result files on the target server.

Operations Tools has three main submenus: jstack, Dump, and Arthas.

8.1 Save jstack Output

Select a cluster node and collect its thread stack. Save the record to make it available for download. Records that are incomplete when the tool restarts are marked as failed.

The jstack form selects a cluster and node for thread stack collection.

The task view reports the jstack collection status and saved result.

Download or view the results and search their contents as needed.

Saved jstack output can be opened or downloaded for searching and review.

8.2 Save Dump Files

Select the cluster, node role, and node. Options include heap or thread dumps, heap dump compression, thread lock information, the remote directory, and a filename prefix. After the task completes, download the output or delete its record.

The Dump form configures the cluster, node, dump type, compression, lock information, and output location.

8.3 Diagnose with Arthas

Select the target DataNode and collection duration, then start a flame graph task. You can stop collection while it is running. After completion, view or delete local flame graph files in the task history.

The Arthas collection form selects a DataNode and the flame graph collection duration.

The flame graph task view shows collection progress and provides access to completed results.

9 Cluster Health Checks

9.1 Health Check Configuration

Open Cluster Health Checks → Health Check Configuration. Select metrics from templates to create a health check plan and configure scheduled execution and notifications.

The health check configuration page lets you create a plan and configure its schedule and notifications.

Open Health Check Templates and select the required templates.
Health check templates provide reusable sets of metrics to include in a plan.
Click Create Health Check Plan, enter a name, and select a cluster.
Select a health check template or click Add Metric.
Configure the time range, conditions, thresholds, mode, and duration for each metric.
Configure the execution schedule and notifications, then click Save Plan.
The saved plan view shows its configured metrics, thresholds, schedule, and notification settings.
Run a manual check to verify that all metrics return results.
A manual run displays the latest health check results for the configured plan.

9.2 Report History

In Report History, filter reports by plan, cluster, and time range. Archive reports before version upgrades, major changes, and project acceptance.

Success: The check items ran and generated results.
Failure: Check the failed metrics and target nodes, and investigate using the tool logs.
Missing data: Check monitoring collection, node connectivity, and the time range.

Report History filters saved reports by plan, cluster, and time range and displays each report status.

9.3 Health Check Templates and Import/Export

Use Health Check Templates to maintain reusable metrics. Export template metrics to reuse them in other environments.
Use Import Plan to import an existing health check plan. After importing, verify that its cluster, metrics, and thresholds are appropriate for the current version.
Deleting a plan does not automatically delete historical reports. Deleting a report affects only the report record and download file saved by the tool.

10 Troubleshooting

Symptom	Check First	Suggested Action
Tool cannot be opened	Process, `APP_ADDR`, port `8386`	Check the PID and logs, and confirm that the port is available.
Server connection fails	IP address, SSH port, credentials, firewall	Test network connectivity, then SSH; verify account permissions.
One-Click Deployment is interrupted	Failed step, disk space, permissions	Check deployment records and the day's application logs, correct prerequisites, and retry.
Workbench is missing from the list	Deployment result and list refresh	Refresh the list and check the service process and port.
Monitoring has no data	Collection targets, connections, time range	Check Prometheus targets, Grafana data sources, and node clocks.
Health check report fails	Plan metrics, selected cluster, collection status	Confirm that the plan is saved and the cluster is available.