Storage Engine
On the Storage Engine screen, you can check the memory and disk usage of Rowstore and Columnstore, the two ways SingleStore stores data. In a distributed environment, skew—where data piles up on a particular node—leads to performance degradation, and this screen shows the distribution across nodes at a glance to help you quickly find skew. The intended readers are DBAs who operate SingleStore.
Rowstore and Columnstore
SingleStore stores data in two ways. Knowing these two concepts first makes the screen easier to read.
- Rowstore stores data in memory by row. It is good for fast reads and writes but uses a lot of memory. Simply put, it is closer to keeping the items you use often on top of your desk.
- Columnstore stores data on disk by column in compressed form. It is good for large-scale analysis and has high disk efficiency. It is closer to compressing items you rarely use and putting them in a warehouse.
The screen shows these two separately.
Summary information
When you select the cluster to query (for example, memsql_cluster) at the top of the screen, it shows the state of Rowstore and Columnstore in two summary cards. Each card shows a WARN badge when the state needs attention.
| Card | Description |
|---|---|
| Rowstore summary information | Total in Memory, Max Node Mem, Avg Memory Deviation, Worst skew table |
| Columnstore summary information | Total on Disk, Compression, Avg Disk Deviation, Worst skew table |
Avg Memory Deviation and Avg Disk Deviation express the usage deviation across nodes as a multiple. The larger the value, the more data is concentrated on a particular node. Worst skew table also points out the table with the most severe skew.
Rowstore area
Since Rowstore uses memory, memory usage and per-node distribution are the key metrics.
| Widget | Description |
|---|---|
| Memory usage by node | Displays per-node host memory usage as a gauge |
| Memory Trend | The time trend of Table Memory (Rowstore) and Server Memory (Process) |
| Memory-Risk Tables (TOP 5) | The top 5 tables with high memory risk |
The per-node memory gauge shows host memory usage together with the usage and total capacity (for example, 2.4 / 3.7 GB) and the percentage. It distinguishes the <80%, 80-90%, and >90% ranges by color, and aggregator (MA/CA) nodes are collapsed by default and can be expanded to view.
Memory-Risk Tables (TOP 5) is made up of the following columns.
| Column | Description |
|---|---|
Table | The table name and database |
Worst Node | The node with the highest usage |
Size (Total) | The total memory usage and row count |
Parts | The number of partitions |
Skew | The skew multiple across nodes |
Columnstore area
Since Columnstore uses disk, disk usage and segment state are key.
| Widget | Description |
|---|---|
| Disk usage by node | Displays per-node disk usage as bars |
| Segment Group trend | A daily comparison of the number of segment groups (group_cnt) and the top tables |
| Size Top 5 | The top 5 tables that take up the most disk |
The per-node disk bar shows the disk SingleStore uses, divided by area. Clicking a bar lets you check the detailed composition per node.
| Area | Description |
|---|---|
Data | Columnstore data |
Snapshot | Snapshot |
Log | Transaction log |
Temp | Temporary data |
Etc | Areas outside partitions |
Others | Other OS usage |
Free | Remaining space |
Size Top 5 shows, per table, the compressed size (Size (Total)), the compression ratio (Compression), the row count (Rows), and the node with the highest usage (Worst Node).
A part that can be confusing here is the Segment Group trend. SingleStore manages Columnstore data as segments, and a background merger combines small segments to tidy them up. The Segment Group trend compares the cluster's group_cnt sum against the day before and shows it, and provides the top 5 tables with the largest current group_cnt in a table together with their 24-hour change (Δ24h) and partition count. If group_cnt keeps trending upward, it can be seen as a sign that the automatic merger is not keeping up with the INSERT rate, in which case you would consider an OPTIMIZE operation.
Checking data skew
In a distributed database, when data is skewed toward a particular node, only that node ends up bearing the load. The Storage Engine screen shows skew with the following metrics.
| Metric | Description |
|---|---|
Max Node Mem | The highest memory usage among the nodes |
Avg Memory Deviation | The average deviation (multiple) of Rowstore memory usage across nodes |
Avg Disk Deviation | The average deviation (multiple) of Columnstore disk usage across nodes |
Skew | The skew multiple across nodes at the table level |
Worst skew table | The table with the most severe skew |
Clicking a row of a table with severe skew opens the detail drawer, where you can check the per-node distribution along with the OPTIMIZE recommendation.
OPTIMIZE recommendation
In the table detail drawer, it guides you step by step on whether data cleanup (OPTIMIZE) is needed. The guidance is divided according to the recommendation level.
Query point
The screen uses the real-time time selector. It shows the latest collected values according to the collection cycle, and by default Memory Trend queries the last 1 hour and the cluster summary queries the last 7 days.