Read log

read-log is a tool for reading/monitoring logs of specific batch. By default, it would keep tracking for logs until the batch is finished.

Usage

usage: livy read-log [-h] --api-url API_URL [--keep-watch | --no-keep-watch]
                     [-v | -q] [--highlight-logger NAME [NAME ...]]
                     [--hide-logger NAME [NAME ...]] [--pb | --no-pb]
                     [--log-file [XXXX.log] | --no-log-file]
                     [--log-file-level {DEBUG,INFO,WARNING,ERROR}]
                     N

Read livy batch execution log from server

positional arguments:
  N                     Livy batch ID for fetching logs

optional arguments:
  -h, --help            show this help message and exit

livy server configuration:
  --api-url API_URL     Base-URL for Livy API server

actions:
  --keep-watch          Keep watching this batch until it is finished
  --no-keep-watch       Only read log once

console:
  -v, --verbose         Enable debug log.
  -q, --silent          Silent mode. Only show warning and error log.
  --highlight-logger NAME [NAME ...]
                        Highlight logs from the given loggers. This option
                        only takes effect when `colorama` is installed.
  --hide-logger NAME [NAME ...]
                        Do not show logs from the given loggers.
  --pb, --with-progressbar
                        Convert TaskSetManager's `Finished task XX in stage Y`
                        logs into progress bar. This option only takes effect
                        when `tqdm` is installed.
  --no-pb, --without-progressbar
                        Not to convert TaskSetManager's logs into progress
                        bar.

file logging:
  --log-file [XXXX.log]
                        Output logs into log file. A temporary file would be
                        created if path is not specificied.
  --no-log-file         Do not output logs into log file.
  --log-file-level {DEBUG,INFO,WARNING,ERROR}
                        Set minimal log level to be written to file. Default:
                        DEBUG.

Configurations

Following configs could be set via Configure command:

root.api_url

URL to Livy server

read_log.keep_watch

To keep watching or not by default. Could be override by --keep-watch and --no-keep-watch argument.