Skip to main content

Metrics loading

Let's learn about the commands that load metrics with the MXQL syntax.

CommandFunction
TAGLOADIt is used to search the data of the category that stores the collected data in the tag-field format.
FLEXLOADIt is used to search the data of the category that stores the collected data in the field format.
LogSinkUsed to query raw log data.
LogSinkCountUsed to aggregate log counts by time unit.

TAGLOAD​

It is used to search the data of the category that stores the collected data in the tag-field format.

OptionFunction
{backward : true}Loads data in reverse chronological order.
{filter : {key:fieldName, value :fieldValue}}Extracts data where the value of the fieldName field is equal to fieldValue.
{filter : {key:fieldName, exclude :fieldValue}}Data is extracted while excluding the data whose fieldName value is equal to fieldValue.
{filter : {key:fieldName, like :fieldValue}}Extracts data where the value of the fieldName field has fieldValue as a substring.
{filter : {key:fieldName, notlike :fieldValue}}Data is extracted while excluding the data whose fieldName value has fieldValue as a substring.

  • In case no options are set

    CATEGORY app_counter
    TAGLOAD

  • In case the backward option is set

    CATEGORY app_counter
    TAGLOAD {backward : true}

  • In case the value filter is set

    CATEGORY app_counter
    TAGLOAD {filter : {key:pid, value:905}}

  • In case the exclude filter is set

    CATEGORY app_counter
    TAGLOAD {filter : {key:pid, exclude:905}}

  • In case of setting the like filter

    CATEGORY app_counter
    TAGLOAD {filter : {key:okindName, like:keeper}}

  • In case of setting the notlike filter

    CATEGORY app_counter
    TAGLOAD {filter : {key:okindName, notlike:keeper}}

  • In case of setting multiple filters

    CATEGORY app_counter
    TAGLOAD { filter:[{key:'host_ip', exclude:'192.168.1.102'}, {key:'container', like:'gateway'}] }
Caution
  • TAGLOAD and FLEXLOAD have set values for CATEGORY that can be set respectively.
  • When using the filter-like or filter-notlike option, if a number comes as a value, it must be enclosed in single quotation marks ('') or double quotation marks ("") for operation. 
    CATEGORY app_counter
    TAGLOAD { filter:[{key:'host_ip', exclude:'192.168.1.102'},{key:okindName, like:"1"}] }

FLEXLOAD​

It is used to search the data of the category that stores the collected data in the field format.

OptionFunction
{backward : true}Loads data in reverse chronological order.

Data is loaded depending on the information set in the data search condition step.

CATEGORY event_cache
FLEXLOAD {backward : true}
Caution

Most categories are using TAGLOAD. FLEXLOAD is used only when using data of categories included in the following.

List of categories that need to use FLEXLOAD​

  • agent_list category

    CATEGORY agent_list
    FLEXLOAD
    SELECT

  • db_agent_list category

    CATEGORY db_agent_list
    FLEXLOAD
    SELECT

  • agent_count category

    CATEGORY agent_count
    FLEXLOAD
    SELECT

  • event_cache category

    CATEGORY event_cache
    FLEXLOAD
    SELECT

LogSink​

Used to query raw log data.

CATEGORY AppLog
LogSink

Log Filtering​

Use LogTag to filter logs by specific conditions.

CATEGORY AppLog
LogTag {key:city, value:"junju"}
LogTag {key:status, value:"200", exclude:true}
LogSink

LogTag Parameters​

  • key: Index key to search
  • value: Value to search (wildcards supported)
  • exclude: Whether to exclude from search (true/false)

Important Notes​

  • LogTag must be written before LogSink or LogSinkCount
  • LogTag can be specified multiple times for the same key or different keys
  • LogTag should be added per search keyword

Example​

Example of querying the count of logs containing specific keywords.

CATEGORY AppLog
LogTag {key:content, value:"*DebugUtil*", exclude:false}
LogTag {key:content, value:"*HttpServletRequest*", exclude:true}
LogSink
SELECT
GROUP {timeunit:1d, pk:pcode, merge:[rows]}
RENAME {src:_rows_, dst:rows}
CREATE {key:_pk_, value:pcode}
UPDATE {key:rows, value:sum}

Query Explanation​

  • First LogTag: Search for logs with "DebugUtil" in the content field
  • Second LogTag: Exclude logs with "HttpServletRequest" in the content field
  • GROUP with timeunit:1d aggregates by day
  • merge:[rows] converts log count to MetricValue type
  • UPDATE with sum calculates daily total count

LogSinkCount​

Used to aggregate log counts by time unit.

Basic Usage​

CATEGORY AppLog
LogSinkCount

Log counts are aggregated by time period (e.g., 1949, 1625, 1674... counts).

Group Aggregation​

You can aggregate logs by grouping based on specific fields.

Single Field Grouping​

CATEGORY AppLog
LogSinkCount {group:oname}

Multiple Field Grouping​

CATEGORY AppLog
LogSinkCount {group:"oname,oid"}

You can specify multiple fields separated by commas.

Hourly Group Aggregation​

You can perform aggregation operations on numeric fields.

CATEGORY AppLog
LogSinkCount {group:oname, aggr:price.n}
Note

Fields that can be stored in Aggr are limited to numeric (n) fields.

Multiple Field Aggregation​

CATEGORY AppLog
LogSinkCount {group:oname, aggr:"price.n,age.n"}

Multiple fields can be entered in aggr separated by commas, but calculations are performed separately for each.

Using with LogTag​

You can aggregate only logs that meet specific conditions using LogTag.

CATEGORY AppLog
LogTag {key:level, value:"ERROR"}
LogSinkCount {group:oname}

The above example filters only ERROR level logs and aggregates counts by oname.

Last updated on