Skip to main content

Log setting

You can configure the log monitoring related settings in Log Configuration. Using the tab at the top, you can use menus for checking the agent settings, determining whether to enable log monitoring, setting the log data retention period and lookup password, registering the log parser, and setting the quick indexes.

Note
  • To use the Activate log monitoring function, the Edit project role is required.

  • The Log edit role allows you to modify the Log Configuration menu other than Activate log monitoring.

Starting the log monitoring

At the top, select the Getting started with log monitoring tab. If you select the WhaTap Docs icon View guide icon and the View plans button, the corresponding guide screen appears.

Set up the agent and enable log monitoring

Log monitoring data setting

In the number 2 area, you can see the Log usage. You can also change the Log retention period and Log lookup password settings.

Log retention period

This is the default data retention period to be applied commonly. If not specified, the default value is 1 day. In addition to the option to select a log retention period, you can enter a desired period of time. If not set separately in the Log usage list, this log data retention period is applied by default. If you set the data retention periods for each category in the Log usage list and select Initialized, the default data retention period is reset.

Log usage amount

You can specify the log data retention periods for each category in the Log usage list. Log Count means the number of logs stacked for the period. For example, Daily log count is the number of logs accumulated during the day, and Expected log count is the number of logs of the today's log count multiplied by the days of data retention.

You can specify the log data retention period as follows. You can free the space by deleting old data according to the specified period.

  • Trial Project

    You can select 1, 2, or 3 days for the data retention period.

  • Paid Project

    For the data retention period, you can select one of 1 day, 2 days, 3 days, 4 days, 5 days, 6 days, 7 days, 10 days, 30 days, and 40 days.

  • Data size-based billing

    The price differs depending on the data retention period.

    For example, if an average of 2 million logs per day accumulates and the data retention period is set to 3 days, an average of 6 million logs is kept on the collection server and subject to billing.

Log lookup password

To enhance security, set the Log lookup password. The log lookup password is optional. If you are using a log lookup password, enter the password to go to the log screen.

Note

In case you forgot the password

If you have the Edit Log role, you can modify it with a new password in Log Configuration.

Log primary parser setting

Select the Log primary parser setting tab at the top of Log Configuration to register parsers for collected logs.

The log primary parser provides GROK and JSON parsers, essential for aggregating log counts by type and searching for specific logs. It extracts search keys and values that match pattern conditions, and the parsed keys are used for log type classification and search indexing.

  • GROK: This parsing is based on the regular expression by default. It provides parsing based on the reserved keywords.

  • JSON: Batch parsing is provided for the JSON output parts of the logs.

Tip

In case of no parsing logic registered, keys for search

category, oid, oname, okind, okindName, @txid, @login, httphost

Reserved words that cannot be registered for the parser

timestamp, message, pcode, category, content, logContent

  • For the following reserved words, indexes are not created even if a parser has been registered.
Note

For more information about the log parser, see the following.

Setting item

Set valueDescriptionETC
CategoryCategory to apply the pattern torequired
Log detection conditionEnter the search key and search value to be applied as the filter. The pattern is applied only to the log data that meets the log detection conditions. If you do not enter any log detection condition, the pattern is applied to all logs.optional
PatternIt is the pattern to parse the log for. Parsing is performed according to the created pattern, and indexes are created. It supports GROK, regular expression syntax.required

Parser list

You can search for registered parsers, add, or edit them. Parsers are evaluated in registered order, and only the first matching parser is applied.

  • If you select + Add on the upper right, the Parser Add window appears.

  • You can change the order of parser settings by dragging the icon in the Priority column of the parser list.

  • Through the parser list's Icon Enable toggle button, you can set whether or not to enable the parser.

  • Through the parser list's Edit icon Edit and Delete icon Delete icons, you can modify or delete the registered parser.

Parser registration order

The following shows the common parser registration procedure.

  1. In Log primary parser setting, select + Add.

  2. In the Parser Add window, select a Parser.

    Note

    Parser and pattern registration method

  3. When you select a parser, the Register pattern button is enabled. After clicking Register pattern, enter the pattern and log in the Parser Simulation window.

    a. Enter the Pattern.

    b. Enter the Log Category and simulation parsing log. Click the Log Search button, then click the Select button to the right of the desired log in the search results to auto-fill the log.

    Note

    Through Log Search, you can select a sample log from collected logs. The selected log is automatically reflected in the simulation input field, allowing you to test immediately without copying and pasting.

    c. To check whether the pattern is normal, click Simulation and then measure the performance of simulation and pattern.

    Note

    For more information on simulation and performance measurement, see the following.

  4. After confirming the simulation result parses successfully, click Apply Pattern.

  5. Click Add to add the parser.

Note

Personal information de-identification

If the search key name is specified as search-key.p during parser registration, de-identification is applied by default (e.g. myname.p **). WhaTap recommends managing de-identification in the De-identification of personal information tab. For more information, see Personal information de-identification.

GROK parser pattern registration

The default syntax is %{SYNTAX:SEMANTIC}. For more information about the GROK parser, see the following. Pattern registration and simulation are required.

  • SYNTAX

    GROK definition pattern.

  • SEMANTIC

    It is the key allocated to the parsed data.

    Note

    It is recommended to use combination words in SEMANTIC so that reserved words are not used.

Registering the JSON format parser pattern

If all or part of a log is output in JSON format, you can parse the JSON output through the JSON format parser. To detect the JSON output of the log, the prefix and postfix options are combined to specify which part of the log to be recognized by JSON for parsing. For more information about the JSON parser, see the following. Simulation is required.

OptionDescription
PrefixSpecify the string before the beginning of JSON string. If not specified, it is identified as a JSON string from the beginning of the log output.
PostfixSpecify the string after the end of JSON string. If not specified, it is identified as a JSON string up to the end of the log output.
IgnoreSpecify fields in the JSON output to exclude from key extraction.
  • Registration example

    Log
    [2022-10-25 10:15:34:145]...(line feed)
    Request : {"key1":"value1","key2":"value2",...}(line feed)
    Response : {"key3":"value3","key4":"value4",...}

    As in the example, to parse both Request JSON and Response JSON for incoming logs, register the following two patterns.

    • Pattern for request parsing

      Strings between "Request : " and "Response" {"key1":"value1","key2":"value2",...}

    • Pattern for response parsing

      Strings from "Response : " to the end of a log {"key3":"value3","key4":"value4",...}

  • JSON custom pattern registration

    If part of a log is output in JSON format, the JSON output can be parsed by a dedicated custom parser. Enter the pattern as follows:

    io.whatap.logsink.parser.JsonFormatParser{}

    To detect the JSON output of the log, the prefix and postfix options are combined to specify which part of the log to be recognized by JSON for parsing.

    Specify an option in {} of JsonFormatParser{}.

    • Registration example

      Log
      [2022-10-25 10:15:34:145]...(line feed)
      Request : {"key1":"value1","key2":"value2",...}(line feed)
      Response : {"key3":"value3","key4":"value4",...}

      As in the example, to parse both Request JSON and Response JSON for incoming logs, register the following two patterns.

      • Pattern for request parsing

        Strings between "Request : " and "Response" {"key1":"value1","key2":"value2",...}

      io.whatap.logsink.parser.JsonFormatParser {prefix:"Request : ",postfix:"Response"}
      • Pattern for response parsing

        Strings from "Response : " to the end of a log {"key3":"value3","key4":"value4",...}

      io.whatap.logsink.parser.JsonFormatParser {prefix: "Response : "}

Parser simulation and performance measurement

After the parser simulation, you can register a pattern. Performance measurement measures the time taken by the parser to perform repeated parsing on the target string for simulation.

  1. Enter values for Pattern and Log. Click the Log Search button, then click the Select button to the right of the desired log in the search results to auto-fill the log.

  2. Click Simulation to check whether the parsing has been successful with the pattern to register.

  3. If the simulation is successful, you can view Simulation result and Performance measurement results.

  4. When you click Apply pattern after simulation, the pattern that has been entered for the selected parser is applied.

Parsing success

If a key is generated by registering a parsing logic, the value parsed with the key is added upon log inquiry. As in the following Live Tail menu's example, the parsed key and value are added.

Parsing Success

The parsed key can be checked in Live Tail, Log Search, and Log Trend.

Log secondary parser setting

Select the Log secondary parser setting tab at the top of Log Configuration to register log parsers.

The log secondary parser processes the values extracted by the primary parser (GROK or JSON) to generate statistics data or new fields. In addition to the HTTP status code-based statistics parsers, it provides various processors such as keyword filters, field processing, response time classification, and GeoIP conversion.

Note

The log secondary parser provides the special purpose secondary parsing function for the primary parsed results. To use the secondary parser, the primary parser must have been registered.

Parser list

Log Secondary Parser List

You can search for registered parsers, add, or edit them.

Adding parsers, changing order, enabling/disabling toggles, editing, and deleting work the same as in the primary parser list.

Parser registration order

The following shows the common parser registration procedure.

Log secondary parser registration order

  1. If you select + Add, the Parser Add window appears.

  2. In the Parser selection window, select a parser. For more information about each parser's setting items, see the following.

  3. Enter the Status codes to exclude.

  4. In the Category selection window, select a category or enter it.

  5. Select Log detection condition or enter it.

  6. Select Add to register a parser.

4xx, 5xx status code parser setting item

The 4xx, 5xx status code parser generates 4xx and 5xx count data based on the status values parsed by the primary parser. To exclude specific status codes, enter the status codes to exclude.

Setting item

Set valueDescriptionETC
CategoryIt is the category to generate the 4xx, 5xx count data.required
Log detection conditionEnter the search key and search value to be applied as the filter. The 4xx, 5xx count data is generated only for the log data that meets the log detection condition. If you do not enter any log detection condition, the data is generated for all logs.optional
Status codes to excludeThe status codes to exclude upon generation of statistics data. If no entry, the 4xx, 5xx count data is generated for all error status codes that correspond between 4xx and 5xx.optional

Registration example for the status parser

GROK Parser - Status parsing pattern registration

If the incoming log is {"msg":"message","status":404} and its status is parsed by the GROK parser as shown in the example, it is parsed as status: 404. If you confirm that the status has been normally parsed, register the status codes to exclude by the 4xx, 5xx status code parser.

If all parsers have been registered, go to Integrated Flex Board and then create the Log 4XX, 5XX count widget.

Log flex board widget - 4xx5xx

If the widget is created, you can see the following data.

Log flex board widget chart - 4xx5xx

  • avg: Average value of the data during the query period

  • max: Maximum value of the data during the query period

  • recently: Final value of the data during the query period

Status code success rate parser setting item

The Status code success rate parser generates HTTP request success rate data based on the status values parsed by the primary parser. To exclude specific status codes, enter the status codes to exclude. For status parsing, see 4xx, 5xx status code parser setting item.

Setting item

Set valueDescriptionETC
CategoryIt is the category to generate the request success rate data.required
Log detection conditionEnter the search key and search value to be applied as the filter. The request success rate data is generated only for the log data that meets the log detection condition. If you do not enter any log detection condition, the data is generated for all logs.optional
Status codes to excludeThe status codes to exclude upon generation of request success rate data. If no entry, the request success rate data is generated for all success rate status codes that correspond between 2xx and 3xx.optional

Data Search

If all parsers have been registered, go to Integrated Flex Board and then create the log request success rate widget.

Flex Board - Log Request Success Rate Widget Template

If the widget is created, you can see the following data.

Log flex board request success rate widget chart-4xx5xx

The data above the chart represents statistics for the lookup period. You can select the statistical method with the latest value, maximum value, or average value. The latest value is selected by default.

Secondary parser processors

In addition to 4xx, 5xx status code parser and Status code success rate parser, the log secondary parser provides the following field processing processors.

Log secondary parser processor list

ProcessorDescription
Contains all keywords filterAdds a flag field to logs that contain all of the specified keywords.
Contains any keyword filterAdds a flag field to logs that contain at least one of the specified keywords.
Rename fieldRenames an existing field name to a different name.
Status code group classificationAutomatically classifies HTTP status codes into 2xx/3xx/4xx/5xx groups.
URL static/dynamic classificationClassifies a URL as static content if it has an extension (.), and as a dynamic servlet if it does not.
URL parameter removalStores the value with the query parameters after ? removed from the URL.
Response time classificationAutomatically classifies the response time (ms) into fast/normal/slow/critical ranges.
JSON field extractionExtracts specific keys from a JSON string field and stores them as new fields.
GeoIP conversionConverts an IP address into country/city information.
Custom inputRegisters a processor directly with an FQN class name.

Contains all keywords filter

Adds a flag field to logs that contain all of the specified keywords.

Set valueDescriptionDefault
Input fieldThe field name to search for keywordscontent
Storage fieldThe field name to store the resulthas_str_all
keywordThe list of keywords to search (enter separated by commas)required

Usage example: If you set the input field to content and the keywords to error, timeout, database, the has_str_all field is added only to logs that contain all three keywords.

Contains any keyword filter

Adds a flag field to logs that contain at least one of the specified keywords.

Set valueDescriptionDefault
Input fieldThe field name to search for keywordscontent
Storage fieldThe field name to store the resulthas_str_any
keywordThe list of keywords to search (enter separated by commas)required

Usage example: If you set the input field to content and the keywords to reboot, shutdown, restart, the has_str_any field is added to logs that contain at least one of the keywords.

Rename field

Renames an existing field name to a different name. If the target field name already exists, it is not overwritten.

Set valueDescriptionDefault
Source fieldThe existing field name to renamerequired
New field nameThe field name after renamingrequired

Status code group classification

Automatically classifies HTTP status codes into the status_2xx, status_3xx, status_4xx, and status_5xx groups.

Set valueDescriptionDefault
Input fieldThe field name that contains the status codestatus
Storage fieldThe field name to store the group classification resultstatus_nxx

Usage example: If the status value is 404, status_4xx is stored in the status_nxx field.

URL static/dynamic classification

Classifies a URL as static content if it contains an extension (.), and as a dynamic servlet if it does not.

Set valueDescriptionDefault
Input fieldThe field name that contains the URLurl
Storage fieldThe field name to store the classification resulturl_type

URL parameter removal

Stores only the path with the query parameters after ? removed from the URL.

Set valueDescriptionDefault
Input fieldThe field name that contains the URLurl
Storage fieldThe field name to store the parameter removal resulturl_without_params

Usage example: /api/v1/users?page=1&size=20/api/v1/users is stored in the url_without_params field.

Response time classification

Automatically classifies the response time (ms) by range. You can set the range thresholds directly.

Set valueDescriptionDefault
Input fieldThe field name that contains the response time (ms)elapsed
Storage fieldThe field name to store the classification resultresponse_class
Fast upper limit (ms)The upper limit of the fast range100
Normal upper limit (ms)The upper limit of the normal range500
Slow upper limit (ms)The upper limit of the slow range2000

The classification results based on the default values are as follows.

Response timeClassification
0 ~ 100msfast
101 ~ 500msnormal
501 ~ 2000msslow
2001ms or morecritical

Usage example: If you filter by response_class: critical in the log explorer, you can query only the logs whose response time exceeds 2 seconds.

JSON field extraction

If a field extracted by the primary parser contains a JSON string, this extracts specific keys from that JSON and stores them as new fields.

Set valueDescriptionDefault
Input fieldThe field name that contains the JSON stringpayload
Keys to extractThe list of JSON keys to extract (enter separated by commas)required
Storage prefixThe prefix to prepend to the extracted key names (e.g., json_)optional
Note
  • If the entire log is JSON, the primary JSON parser parses it automatically. This processor is useful for additionally parsing the JSON part contained within a text log when using the GROK primary parser.

  • To protect performance, parsing is skipped if the JSON field size exceeds 10 KB.

Usage example

Primary GROK parsing result
payload = {"user":"alice","action":"purchase","product":"laptop","amount":45000}

If you set the input field to payload, the keys to extract to user, action, and the storage prefix to pay_, they are extracted as follows.

JSON field extraction result
pay_user = alice
pay_action = purchase

GeoIP conversion

Converts an IP address into country and city information. It uses the IP2Location database to enable region-based analysis of access logs.

Set valueDescriptionDefault
Input fieldThe field name that contains the IP addressclient_ip
Storage prefixThe prefix of the result field namesgeo

As a result, two fields, {prefix}_country and {prefix}_city, are created.

Usage example: If the input field client_ip value is 223.130.195.200 and the storage prefix is geo:

GeoIP conversion result
geo_country = KR
geo_city = Seongnam

Custom input

You can register a processor that is not in the list directly with an FQN (Fully Qualified Name) class name.

Custom input format
io.whatap.logsink.processor.ClassName{"param1":"value1","param2":"value2"}
Note

The class name and parameter format must be accurate. If the input is incorrect, the processor may not work.

Secondary parser chaining

You can register multiple secondary parsers in the same category and chain them. Each processor runs in the order of registration.

Original log
2026-04-23T10:30:00.123+0900|INFO|api-01|192.168.1.10|/api/v1/orders?detail=true|200|1234|{"user":"alice","action":"purchase"}|Request processed
OrderParserInput → Output
PrimaryGROKtimestamp, level, host, client_ip, url, status, elapsed, payload, content
Secondary-1JSON field extractionpayload → user, action
Secondary-2Response time classificationelapsed → response_class (slow)
Secondary-3Status code group classificationstatus → status_nxx (status_2xx)
Secondary-4URL parameter removalurl → url_without_params (/api/v1/orders)
Secondary-5GeoIP conversionclient_ip → geo_country (KR), geo_city (Seoul)
Note

The Category of the secondary parser must be set identically to that of the primary parser. If you set a different category, the secondary parser is not applied to that log.

Fast Index Setting

Select the Fast index setting tab at the top of Log Configuration. Collecting a large number of logs can significantly decrease the log search performance. The frequently used search conditions are created as index, you can improve the log search performance for quick search. The setting items are as follows:

Set valueRequiredDescription
CategoryMandatoryCategory to be set as fast index
Search KeyMandatorySearch key for fast index setting
Case insensitiveOptionWhether to be case sensitive
RuleMandatory* must be included at least one.
EnabledMandatoryActive or inactive (default value is true)

Importing/exporting log settings

You can save parser settings, Fast index setting, and filter settings in JSON file format, and apply them by importing a JSON file from other projects. You can reduce the hassle of creating the settings repeatedly for each project.

  1. Add parser settings, Fast index setting, and filter settings to one project.

  2. Select JSON Export icon on the upper right of each configuration tab.

  3. On the upper right of the Export JSON window, select Export.

  4. The JSON configuration file is saved on your PC.

  5. Move to another project and then go to the Log > Log setting menu.

  6. Select the configuration tab where you have exported the JSON configuration file before, and then select Import icon.

  7. If the file selection window appears, select the JSON configuration file saved on your PC.

  8. If the Import JSON window appears, check the configuration file and then select Add to list or Overwrite.

  9. Select Save on the upper right of the screen.

Caution

After importing the JSON configuration file, you cannot save the imported settings unless you select Save.

Log long-term archive statistics

Select the Log long-term archive statistics tab at the top of Log Configuration. Log data is so large and difficult to retain for a long time. Using the Set log statistics data function, you can save information on how many logs that meet specific conditions are collected every 5 minutes. Even if actual log data has been deleted for a long time, you can check the trend of how many logs that meet the conditions are collected.

Adding the log long-term archive statistics

Log long-term archive statistics

If you select + Add under the Log long-term archive statistics tab, the Log long-term archive statistics Add window appears. You can add rules by using + Add or delete the created rules by using the - icon.

Setting item

FieldDescription
CategoryCategory to apply the rule to.
Statistic KeyThe same key cannot be set twice to save when a log that meets the rule is generated.
Log detection conditionCondition for generating log statistical data. Statistical data is generated based on how many logs that meet this condition are collected.
ExcludeIf checked, statistical data is generated with values that do not correspond to the entered conditions.
Case SensitiveSpecify case sensitivity for the values of the entered log detection conditions.
EnabledActive or inactive (default value is true)

Example

If a setting is added as follows, statistical data is generated with a key value of TotalCount for the logs whose status is 200 or 300.

Log long-term archive statistics example

Data Search

  1. Create a widget by searching Log long-term archive statistics in Widget templates of Integrated Flex Board.

    Log long-term archive statistics widget template

  2. Enter the Category and Statistic key to view and then select Apply.

    Log long-term archive statistics widget category and key selection

  3. With the added settings, you can check the Log long-term archive statistics data as follows.

    Log long-term archive statistics chart widget

Personal information de-identification

In De-identification of personal information, you can mask personal information in log data or replace it with safe values.

  • Encryption: Specified sensitive data is encrypted and stored in the database.

  • Masking: When displayed on the log screen, sensitive data is masked (***) by default.

  • Substitution: Sensitive data is replaced with a user-specified value that is safe.

Personal information classification

The items classified as personal information are as follows: If there are any items that need to be treated as sensitive data in addition to personal information, mask or replace the desired items in the log data by using the De-identification of personal information feature.

  • National ID number
  • Passport number

  • Driver's license number

  • Foreigner registration number

  • Credit card number

  • Bank account number

  • Biometric information

Required roles

De-identified personal information is displayed as masked. Only users with the log personal information query role can view original data through the Display personal information toggle. Adding or modifying de-identification target search keys requires the log editing role.

Note

For more information about the log personal information query and log editing roles, see the following.

Setting the personal information de-identification

You can protect sensitive information by applying masking to new log data.

Log Data De-identification

  1. In the De-identification of personal information tab, select Add.

  2. In the De-identification of personal information Add pane on the right, select Category and enter the Search key to mask.

  3. Click Add to add the setting.

  4. After checking the de-identified items in the De-identification of personal information list, click Save on the upper right to save the current settings.

    Log Data De-identification List

    • The de-identified search keys are output in the format of search key.pii in the log content.

    • You can perform Toggle enabling iconEnable for de-identified items in the Enable column of the De-identification of personal information list.

    • You can Edit icon modify and Remove icon remove in the edit column of the De-identification of personal information list.

Tip

If the search key name is specified as search-key.p in the Log primary parser setting tab, it is automatically registered as a de-identification target. You can check or disable auto-registered items in Specified search key after selecting the Category in the De-identification of personal information tab.

We recommend configuring de-identification directly in the De-identification of personal information tab rather than using the .p suffix in parser settings.

Personal information display

De-identified data is displayed as masked in the Live Tail, Log Trend, and Log Search menus. Users with the log personal information query role can click the Display personal information Toggle enabling icon button in the log list to view the original data.

Log personal information display sc

Note

The search key specified in the De-identification of personal information tab appears in the log content of the Log menu in the format of Search key.pii.

Example of status.pii ***

Checking the log's personal information de-identification

Caution

If you replace the existing data using the Log's personal information de-identification checking feature, the data cannot be recovered.

Log's personal information de-identification checking identifies personal information in existing log data and replaces it with the Value to substitute specified by the user.

Replacement by target key

If you specify a target key, only the value stored in that key is replaced, and the rest of the log content is kept as is. The target key applies to both the field value and the tag value of a log. If you do not specify a target key, personal information is identified and replaced across the entire log content, the same as before.

Note

Replacement applies only to the stored log data. The original value may remain in the search index even after replacement.

You specify the target key in the Target key input field of the log personal information de-identification check form. On the screen, field and tag are not distinguished and are entered as a single target key.

Log Data De-identification Check

  1. Specify the start time and end time.

  2. Set values for Category .

  3. Select Personal information entity from the examples or Direct Input.

    Note

    When selecting Direct Input, directly enter a regular expression pattern to identify personal information.

  4. Select whether to replace personal information, if it is identified via the Whether to apply substitution toggle.

  5. Enter Value to substitute to change the string identified by the regular expression pattern in the existing log.

    Note
    • Value to substitute must not be longer than the original value.

    • When enabling the Whether to apply substitution toggle, be sure to enter Value to substitute.

  6. Enter the conditions and then select Start inspection.

    Note

    You can stop this step at any time by clicking Stop inspection during the checkup.

  7. When applying substitution, enter Replace in the Start of inspection for personal information de-identification confirmation window, and then click the OK button.

    Checking the log's personal information de-identification

  8. After the inspection is finished, you can see Inspection result on the right as in the example image.

    • Category: It displays the categories checked.

    • Search start time and Search end time: It displays the time range for the checkup.

    • Personal information entity: It displays the personal items that have been checked.

    • Personal information pattern: It displays the regular expression pattern used.

    • Value to substitute: It displays the applied replacement value.

    • Pattern matching result: The number of personal data identified is displayed. You can check the sample logs by clicking Check results.

    • Inspection result: It displays the result where personal information has been identified.

      SUCCESS (check success), FAILED (check failure), STOP (check stop), TIMED_OUT (check ended due to timeout)

Check inspection history

To view the results of past checks after performing de-identification check for the log's personal information, click the Check inspection history button at the upper right. You can trace the substitution history, inspection time, and applied personal information items. You can also download the list information in CSV file format by clicking the Download icon CSV button at the upper right of the Check inspection history window.

Checking the history of log's personal information de-identification

Personal information pattern

Examples of regular expression patterns used to identify personal information in the existing log are as follows:

Personal information patternPersonal information exampleRegular expression patternReplacement value
National ID number123456-1234567
(?<!w)d{2}(0[1-9]|1[0-2])([0-2][0-9]|3[0-1])-[1-4]d{6}(?!w)
******-*******
Passport numberM1234567
([MSRODTC][0-9]{7}\b)
********
Driver's license number01-23-456789-00
(?<!w)d{2}-d{2}-d{6}-d{2}(?!w)
**-**-******-**
Foreigner registration number987654-1234567
(?<!w)d{6}-?[5-8]d{6}(?!w)
******-*******
Credit card number1234-5678-9012-3456
(?<!w)d{4}-d{4}-d{4}-d{4}(?!w)
****-****-****-****
Mobile phone number010-1234-5678
(?<!w)(+82-?1[016789]-?d{3,4}-?d{4}|01[016789]-?d{3,4}-?d{4})(?!w)
***-****-****
Email addressexample@domain.com
([a-zA-Z0-9][a-zA-Z0-9._%+-]*@[a-zA-Z0-9.-]+.[a-zA-Z]{2,})
***@***.***
Bank account number123456-01-654321
(?<!w)d{6}-d{2}-d{6}(?!w)
******-**-******
Business registration number111-11-01111
(?<!w)d{3}-d{2}-d{5}(?!w)
***-**-*****
Corporate registration number111111-1111111
(?<!w)d{6}-?d{7}(?!w)
******-*******