preprocess() in Drain.py has issue about logbert HOT 4 CLOSED

Try this in
process_tbird.sh
--regex "$REGEX1" "$REGEX2" "$REGEX3" "$REGEX4"
data_process.py

parser.add_argument("--regex", nargs='*', help="regex to clean log messages", default='')

from logbert.

Hi,

I found the correct way when I checked the original Drain_demo.py. (https://github.com/logpai/logparser/blob/master/demo/Drain_demo.py)

Regular expression list for optional preprocessing (default: [])
regex = [
r'blk_(|-)[0-9]+' , # block id
r'(/|)([0-9]+.){3}[0-9]+(:[0-9]+|)(:|)', # IP
r'(?<=[^A-Za-z0-9])(-?+?\d+)(?=[^A-Za-z0-9])|[0-9]+$', # Numbers
]
st = 0.5 # Similarity threshold
depth = 4 # Depth of all leaf nodes

parser = Drain.LogParser(log_format, indir=input_dir, outdir=output_dir, depth=depth, st=st, rex=regex)
parser.parse(log_file)

--regex="$REGEX1 $REGEX2 $REGEX3 $REGEX4" is passed the value as 1 size of list.

Please check it.

from logbert.

Thank you for letting me know. I used nargs='*' in argparse to receive multiple inputs as list (check this for usage ). But I should use --regex $REGEX1 $REGEX2 $REGEX3 instead of --regex="$REGEX1 $REGEX2 $REGEX3" in shell scripts

from logbert.

Not covered REGEX3='(?<=Warning: we failed to resolve data source name )[\w\s]+'
argparse is processing it to
['(0x)[0-9a-fA-F]+', '\d+.\d+.\d+.\d+', '(?<=Warning:', 'we', 'failed', 'to', 'resolve', 'data', 'source', 'name', ')[\w\s]+', '\d+']

from logbert.