We use the same configuration files for IoTDB DataNode and Standalone version, all under the conf.
datanode-env.sh/bat：Environment configurations, in which we could set the memory allocation of DataNode and Standalone.
iotdb-datanode.properties：IoTDB DataNode system configurations.
For the convenience of users, IoTDB provides users with hot modification function, that is, modifying some configuration parameters in iotdb-datanode.properties and iotdb-common.properties during the system operation and applying them to the system immediately.
In the parameters described below, these parameters whose way of Effective is hot-load support hot modification.
Trigger way: The client sends the command(sql) load configuration to the IoTDB server.
The environment configuration file is mainly used to configure the Java environment related parameters when DataNode is running, such as JVM related configuration. This part of the configuration is passed to the JVM when the DataNode starts.
The details of each parameter are as follows:
| Name | MAX_HEAP_SIZE |
|---|
| Description | The maximum heap memory size that IoTDB can use |
| Type | String |
| Default | On Linux or MacOS, the default is one quarter of the memory. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G. |
| Effective | After restarting system |
| Name | HEAP_NEWSIZE |
|---|
| Description | The minimum heap memory size that IoTDB will use when startup |
| Type | String |
| Default | On Linux or MacOS, the default is min{cores * 100M, one quarter of MAX_HEAP_SIZE}. On Windows, the default value for 32-bit systems is 512M, and the default for 64-bit systems is 2G. |
| Effective | After restarting system |
| Name | MAX_DIRECT_MEMORY_SIZE |
|---|
| Description | The max direct memory that IoTDB could use |
| Type | String |
| Default | Equal to the MAX_HEAP_SIZE |
| Effective | After restarting system |
| Name | JMX_LOCAL |
|---|
| Description | JMX monitoring mode, configured as yes to allow only local monitoring, no to allow remote monitoring |
| Type | Enum String: "true", "false" |
| Default | true |
| Effective | After restarting system |
| Name | JMX_PORT |
|---|
| Description | JMX listening port. Please confirm that the port is not a system reserved port and is not occupied |
| Type | Short Int: [0,65535] |
| Default | 31999 |
| Effective | After restarting system |
| Name | JMX_IP |
|---|
| Description | JMX listening address. Only take effect if JMX_LOCAL=false. 0.0.0.0 is never allowed |
| Type | String |
| Default | 127.0.0.1 |
| Effective | After restarting system |
We STRONGLY RECOMMENDED you CHANGE the PASSWORD for the JMX remote connection.
The user and passwords are in ${IOTDB_CONF}/conf/jmx.password.
The permission definitions are in ${IOTDB_CONF}/conf/jmx.access.
| Name | dn_rpc_address |
|---|
| Description | The client rpc service listens on the address. |
| Type | String |
| Default | 0.0.0.0 |
| Effective | After restarting system |
| Name | dn_rpc_port |
|---|
| Description | The client rpc service listens on the port. |
| Type | Short Int : [0,65535] |
| Default | 6667 |
| Effective | After restarting system |
| Name | dn_internal_address |
|---|
| Description | DataNode internal service host/IP |
| Type | string |
| Default | 127.0.0.1 |
| Effective | Only allowed to be modified in first start up |
| Name | dn_internal_port |
|---|
| Description | DataNode internal service port |
| Type | int |
| Default | 10730 |
| Effective | Only allowed to be modified in first start up |
- dn_mpp_data_exchange_port
| Name | mpp_data_exchange_port |
|---|
| Description | MPP data exchange port |
| Type | int |
| Default | 10740 |
| Effective | Only allowed to be modified in first start up |
- dn_schema_region_consensus_port
| Name | dn_schema_region_consensus_port |
|---|
| Description | DataNode Schema replica communication port for consensus |
| Type | int |
| Default | 10750 |
| Effective | Only allowed to be modified in first start up |
- dn_data_region_consensus_port
| Name | dn_data_region_consensus_port |
|---|
| Description | DataNode Data replica communication port for consensus |
| Type | int |
| Default | 10760 |
| Effective | Only allowed to be modified in first start up |
- dn_join_cluster_retry_interval_ms
| Name | dn_join_cluster_retry_interval_ms |
|---|
| Description | The time of data node waiting for the next retry to join into the cluster |
| Type | long |
| Default | 5000 |
| Effective | After restarting system |
- dn_target_config_node_list
| Name | dn_target_config_node_list |
|---|
| Description | ConfigNode Address for DataNode to join cluster |
| Type | String |
| Default | 127.0.0.1:10710 |
| Effective | Only allowed to be modified in first start up |
- dn_rpc_thrift_compression_enable
| Name | dn_rpc_thrift_compression_enable |
|---|
| Description | Whether enable thrift's compression (using GZIP). |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
- dn_rpc_advanced_compression_enable
| Name | dn_rpc_advanced_compression_enable |
|---|
| Description | Whether enable thrift's advanced compression. |
| Type | Boolean |
| Default | false |
| Effective | After restarting system |
- dn_rpc_selector_thread_count
| Name | dn_rpc_selector_thread_count |
|---|
| Description | The number of rpc selector thread. |
| Type | int |
| Default | false |
| Effective | After restarting system |
- dn_rpc_min_concurrent_client_num
| Name | dn_rpc_min_concurrent_client_num |
|---|
| Description | Minimum concurrent rpc connections |
| Type | Short Int : [0,65535] |
| Description | 1 |
| Effective | After restarting system |
- dn_rpc_max_concurrent_client_num
| Name | dn_rpc_max_concurrent_client_num |
|---|
| Description | Max concurrent rpc connections |
| Type | Short Int : [0,65535] |
| Description | 65535 |
| Effective | After restarting system |
| Name | dn_thrift_max_frame_size |
|---|
| Description | Max size of bytes of each thrift RPC request/response |
| Type | Long |
| Unit | Byte |
| Default | 536870912 |
| Effective | After restarting system |
- dn_thrift_init_buffer_size
| Name | dn_thrift_init_buffer_size |
|---|
| Description | Initial size of bytes of buffer that thrift used |
| Type | long |
| Default | 1024 |
| Effective | After restarting system |
- dn_core_client_count_for_each_node_in_client_manager
| Name | dn_core_client_count_for_each_node_in_client_manager |
|---|
| Description | Number of core clients routed to each node in a ClientManager |
| Type | int |
| Default | 200 |
| Effective | After restarting system |
- dn_max_client_count_for_each_node_in_client_manager
| Name | dn_max_client_count_for_each_node_in_client_manager |
|---|
| Description | Number of max clients routed to each node in a ClientManager |
| Type | int |
| Default | 300 |
| Effective | After restarting system |
| Name | dn_system_dir |
|---|
| Description | The directories of system files. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/system (Windows: data\datanode\system) |
| Effective | After restarting system |
| Name | dn_data_dirs |
|---|
| Description | The directories of data files. Multiple directories are separated by comma. The starting directory of the relative path is related to the operating system. It is recommended to use an absolute path. If the path does not exist, the system will automatically create it. |
| Type | String[] |
| Default | data/datanode/data (Windows: data\datanode\data) |
| Effective | After restarting system |
| Name | dn_multi_dir_strategy |
|---|
| Description | IoTDB's strategy for selecting directories for TsFile in tsfile_dir. You can use a simple class name or a full name of the class. The system provides the following three strategies:
1. SequenceStrategy: IoTDB selects the directory from tsfile_dir in order, traverses all the directories in tsfile_dir in turn, and keeps counting;
2. MaxDiskUsableSpaceFirstStrategy: IoTDB first selects the directory with the largest free disk space in tsfile_dir;
You can complete a user-defined policy in the following ways:
1. Inherit the org.apache.iotdb.db.storageengine.rescon.disk.strategy.DirectoryStrategy class and implement its own Strategy method;
2. Fill in the configuration class with the full class name of the implemented class (package name plus class name, UserDfineStrategyPackage);
3. Add the jar file to the project. |
| Type | String |
| Default | SequenceStrategy |
| Effective | hot-load |
| Name | dn_consensus_dir |
|---|
| Description | The directories of consensus files. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/consensus |
| Effective | After restarting system |
| Name | dn_wal_dirs |
|---|
| Description | Write Ahead Log storage path. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/wal |
| Effective | After restarting system |
| Name | dn_tracing_dir |
|---|
| Description | The tracing root directory path. It is recommended to use an absolute path. |
| Type | String |
| Default | datanode/tracing |
| Effective | After restarting system |
| Name | dn_sync_dir |
|---|
| Description | The directories of sync files. It is recommended to use an absolute path. |
| Type | String |
| Default | data/datanode/sync |
| Effective | After restarting system |
GC log is off by default.
For performance tuning, you may want to collect the GC info.
To enable GC log, just add a parameter "printgc" when you start the DataNode.
nohup sbin/start-datanode.sh printgc >/dev/null 2>&1 &
Or
sbin\start-datanode.bat printgc
GC log is stored at IOTDB_HOME/logs/gc.log.
There will be at most 10 gc.log.* files and each one can reach to 10MB.
- dn_default_space_move_thresholds
| Name | dn_default_space_move_thresholds |
|---|
| Description | Define the minimum remaining space ratio for each tier data catalogue; when the remaining space is less than this ratio, the data will be automatically migrated to the next tier; when the remaining storage space of the last tier falls below this threshold, the system will be set to READ_ONLY |
| Type | double |
| Default | 0.15 |
| Effective | hot-load |
| Name | remote_tsfile_cache_dirs |
|---|
| Description | Cache directory stored locally in the cloud |
| Type | string |
| Default | data/datanode/data/cache |
| Effective | After restarting system |
- remote_tsfile_cache_page_size_in_kb
| Name | remote_tsfile_cache_page_size_in_kb |
|---|
| Description | Block size of locally cached files stored in the cloud |
| Type | int |
| Default | 20480 |
| Effective | After restarting system |
- remote_tsfile_cache_max_disk_usage_in_mb
| Name | remote_tsfile_cache_max_disk_usage_in_mb |
|---|
| Description | Maximum Disk Occupancy Size for Cloud Storage Local Cache |
| Type | long |
| Default | 51200 |
| Effective | After restarting system |
| Name | object_storage_type |
|---|
| Description | Cloud Storage Type |
| Type | string |
| Default | AWS_S3 |
| Effective | After restarting system |
| Name | object_storage_bucket |
|---|
| Description | Name of cloud storage bucket |
| Type | string |
| Default | iotdb_data |
| Effective | After restarting system |
| Name | object_storage_endpoiont |
|---|
| Description | endpoint of cloud storage |
| Type | string |
| Default | None |
| Effective | After restarting system |
- object_storage_access_key
| Name | object_storage_access_key |
|---|
| Description | Authentication information stored in the cloud: key |
| Type | string |
| Default | None |
| Effective | After restarting system |
- object_storage_access_secret
| Name | object_storage_access_secret |
|---|
| Description | Authentication information stored in the cloud: secret |
| Type | string |
| Default | None |
| Effective | After restarting system |
