lnav

lnav v0.15.0

Features:

• Added the :filter-context command to show lines surrounding filter matches, similar to grep's -C option. The command accepts one or two arguments for the number of messages to show before and after each match. In the LOG view, context is counted in whole messages (including continuation lines). The z/Z keys can also be used to increase/decrease the context by one in the LOG, TEXT, and TIMELINE views. Context lines are styled using the new context-line theme style.
• Added a built-in metrics_log format that recognizes CSV files whose first column header is Time/Timestamp/ts/ Date... and whose subsequent rows begin with a parseable timestamp. Each row is rendered with a timestamp and a <column>=<value> pair for each numeric column. Rows from multiple files that have the same timestamp are merged into a single line. Values are right-aligned and decorated with reverse-video bars proportional to each column's observed min/max range. When the row is focused, an overlay below it labels each column with the source file stem. Exports from Excel, PowerShell, and Grafana should be handled as-is.
• Added the all_metrics SQL virtual table, a long-format view across every open metrics file. The metric column contains the name of the metric and value contains its value.
• The TIMELINE view now supports overlaying metric sparklines at the top of the view. The following commands can be used to manage this:
  • :timeline-metric [<source>.]<metric> picks a column from any loaded metrics file
  • :timeline-metric-sql <label> <query> takes an arbitrary SELECT log_time, value FROM ... that can target any table, including search-table columns.
  • :clear-timeline-metric <label> removes metrics. Up to four metrics can be added.
• Added support for "tabular" formats (e.g. CSV, TSV). The format definition for this type of file sets file-type to tabular and then defines the known columns. When opening a file of this type, the separator will be automatically detected and the header compared against the columns defined in the tabular formats. If a good match is found, it will be used as the format for the file. Quoted cells that span multiple physical lines (an embedded LF inside "...") are stitched back into a single log message, so an Excel-exported CSV with a multi-line free-text column shows one logline per row and SQL/search operates on the merged cell value.
• Added a log format for the fsck_apfs and fsck_hfs tools on macOS, covering both the started/completed lifecycle lines and legacy run entries. This replaces the previous fsck_hfs_log format, which only matched the start lines. The new format exposes device, tool, and action fields, groups messages by device in the TIMELINE view, and highlights error: lines and FILESYSTEM CLEAN status messages.
• Added a log format for the Asterisk PBX framework. The call ID (e.g. C-00000001) is used as the operation ID so the messages for a call are grouped in the TIMELINE view.
• Log format value definitions now accept a unit object with suffix and divisor properties. suffix specifies how numeric fields are humanized. divisor normalizes the raw value to the base unit implied by suffix — e.g. a field storing milliseconds with "suffix": "s" declares "divisor": 1000.
• The details overlay now shows per-column statistics for the focused message. Numeric columns get a min..max of N range summary on the value line, plus a p50/p90/p99 percentile sub-line for columns with enough samples to characterize the distribution shape. Identifier and metrics-text columns get an estimated distinct-value count (~K distinct of N). Distinct counts are computed from a HyperLogLog sketch (~4 KB per text col

lnav v0.14.1

Features:

• Added the :reload-view command, bound to F5/⌘-R, that re-runs the operation that populated the current view. In the DB view, this re-executes the last SQL query; in the TIMELINE view, it rebuilds the index. Views that don't have a meaningful reload report an error.
• The DB view now shows a status bar above the bottom status bar with the SQL query that populated the view, the relative time when it was run, and how long it took. A reload icon on the left side of the bar can be clicked to re-run the query. For queries that read from log-backed tables, the status bar also calls out whether the results are "on current log data" or "on old log data" relative to what lnav is currently indexing. The same information is exposed via the new lnav_views.view_details column.
• For terminals that support the Kitty Keyboard protocol, the following hotkeys are now supported in the prompt:
  • ⌘-C to copy the current selection to the clipboard.
  • ⌘-A to select all text.
  • ⌘-X to cut the current selection to the clipboard.
  • ⌘-Z to undo the last change.
• The multi-line prompt can now be resized with the keyboard: ALT-= grows the prompt by one line and ALT-- shrinks it by one line. This complements the existing click-and-drag resize on the prompt's status bar.
• Additional readline-style key bindings in the prompt (issue #1676):
  • ALT-f / ALT-b move forward/backward by word.
  • ALT-d cuts to the end of the next word.
  • ALT-BACKSPACE is an alias for CTRL-W.
  • ALT-l / ALT-u lower- or upper-case the next word.
  • ALT-c capitalizes the next word.
  • CTRL-h and CTRL-d are aliases for BACKSPACE and DELETE.
  • CTRL-t transposes the two characters before the cursor.
• In the log message details (opened by pressing p), a File: row has been added that lists the file and line number the log message is from. Also, for JSON logs, you can focus on the JSON fields: row and press c to copy the raw log message to the clipboard.
• The measure_with_units collator now recognizes (KiB, MiB, ...).
• Added disfavors to external-editor configuration to express file names that the editor is not interested in handling.
• The lnav_views.view_details column is now populated for every time-based view (log, histogram, spectro, timeline, db) with a JSON zoom-level field reflecting the current :zoom-to setting.
• The :zoom-to command (and the z/Z hotkeys) now affects only the currently focused view; previously, zooming in either the histogram or the spectrogram view would change both views' zoom level in lockstep. Each view (log, histogram, spectro, timeline, db) now has its own independent zoom level.
• The :zoom-to command now accepts + and - as shortcuts to step in or out by one level from the current view's zoom (the same behavior as the z/Z hotkeys).
• SQL statements can now use $zoom_level to refer to the current zoom level in the DB view. Pressing z/Z in the DB view will now rerun the last query with the new zoom value. The stats.hist PRQL function now uses $zoom_level as the default value for the slice parameter.
• A stats.timeseries PRQL function has been added to make it easier to perform an aggregation over buckets of time.

Breaking changes:

• The humanize_file_size() SQLite function now uses 1,000 for the base instead of 1,024.

Bug Fixes:

• A PRQL query can now start with let in interactive mode.
• Fix a bug in file loading that could cause a short read and crash in some situations.
• Fix a lockup when viewing a file that contained log messages and lots of binary data.
• Update Regex101 import functionality to handle a change in their API response.
• More hardening for some diabolical inputs:
  • Unsuppor

lnav v0.14.0

Features:

• The Filter configuration panel in the TUI now supports editing the minimum log level and minimum/maximum times for the current view. Previously, only the :set-min-log-level, :hide-lines-before, and :hide-lines-after commands could be used to set the values. Pressing l will create/set the log level. Pressing m will create/set the minimum time and pressing Shift + M will create/set the maximum time. Live preview has also been added to show which lines will be filtered out when the min/max is applied.
• The -S/--since and -U/--until flags have been added to limit how much of a log file is indexed. Supported values are relative (e.g. "yesterday", "30 min ago") or absolute local times ("2020-01-01"). The values on the main command-line set the defaults for all values that are opened. The :open command supports the same options to change the values from the default for a particular file. Files with content that lie completely outside of the cutoff will be closed to reduce resource usage.
• Broadened support for viewing files on remote hosts. The "tailer" program that is transferred to the remote host to monitor files and perform other tasks has been translated to Python3. If the remote host does not have Python, lnav will fall back to the APE binary.
• Highlights can now be applied to a particular field instead of the whole line. For log formats, a value entry can now have a highlights object that contains definitions for the pattern to match and the style to apply. For interactive use, the :highlight-field command was added.
• The "Files" panel now shows a progress bar for each file as it is being indexed and finishes with a check-mark if indexing was successful, a warning sign if the file has some notes, or an error mark if something else happened.
• The :write-jsonlines-to command now supports the LOG view. When lines are marked in the LOG view, the command will write each marked log message as a JSON object with the standard log fields (log_path, log_time, log_level, log_opid, log_line_link) along with any format-specific parsed values, comments, and tags. The --all flag can be used to write all visible log lines instead of only marked lines.
• Added a log format for MongoDB's structured JSON log format (4.4+).
• Added a log format for the Robot Framework's debug log file format.
• Added "timestamp" as a value kind for log format definitions. Fields with this kind will be parsed and displayed in the local timezone, and SQL queries will return them in a normalized format.
• Added the :toggle-sticky-header command to pin a line to the top of the view as a sticky header. Sticky headers remain visible as you scroll past them, making it easy to keep context lines in view. The :clear-all-sticky-headers command removes all sticky headers in the current view. A hidden log_sticky_mark column is available in log tables to get/set the sticky state via SQL. Sticky headers and user bookmarks are saved and restored across sessions for both log and text views. The command is bound to CTRL+S by default.
• In pager mode (activated by the -q flag), the input will always be written to the terminal if it is smaller than the height. Previously, if the input took awhile to generate, nothing would be written. Also, when the input is larger than the terminal height, any marked lines will be written to the terminal.
• Introducing "Log-Oriented Debugging", a collection of features to streamline mapping log messages back to the source code that generated them. For example, given the log message "Hello, Steve!" and the source directory containing the log statement. lnav can find the line of code that generated the message, such as `logging.info("

lnav v0.14.0

Features:

• The Filter configuration panel in the TUI now supports editing the minimum log level and minimum/maximum times for the current view. Previously, only the :set-min-log-level, :hide-lines-before, and :hide-lines-after commands could be used to set the values. Pressing l will create/set the log level. Pressing m will create/set the minimum time and pressing Shift + M will create/set the maximum time. Live preview has also been added to show which lines will be filtered out when the min/max is applied.
• The -S/--since and -U/--until flags have been added to limit how much of a log file is indexed. Supported values are relative (e.g. "yesterday", "30 min ago") or absolute local times ("2020-01-01"). The values on the main command-line set the defaults for all values that are opened. The :open command supports the same options to change the values from the default for a particular file. Files with content that lie completely outside of the cutoff will be closed to reduce resource usage.
• Broadened support for viewing files on remote hosts. The "tailer" program that is transferred to the remote host to monitor files and perform other tasks has been translated to Python3. If the remote host does not have Python, lnav will fall back to the APE binary.
• Highlights can now be applied to a particular field instead of the whole line. For log formats, a value entry can now have a highlights object that contains definitions for the pattern to match and the style to apply. For interactive use, the :highlight-field command was added.
• The "Files" panel now shows a progress bar for each file as it is being indexed and finishes with a check-mark if indexing was successful, a warning sign if the file has some notes, or an error mark if something else happened.
• The :write-jsonlines-to command now supports the LOG view. When lines are marked in the LOG view, the command will write each marked log message as a JSON object with the standard log fields (log_path, log_time, log_level, log_opid, log_line_link) along with any format-specific parsed values, comments, and tags. The --all flag can be used to write all visible log lines instead of only marked lines.
• Added a log format for MongoDB's structured JSON log format (4.4+).
• Added a log format for the Robot Framework's debug log file format.
• Added "timestamp" as a value kind for log format definitions. Fields with this kind will be parsed and displayed in the local timezone, and SQL queries will return them in a normalized format.
• Added the :toggle-sticky-header command to pin a line to the top of the view as a sticky header. Sticky headers remain visible as you scroll past them, making it easy to keep context lines in view. The :clear-all-sticky-headers command removes all sticky headers in the current view. A hidden log_sticky_mark column is available in log tables to get/set the sticky state via SQL. Sticky headers and user bookmarks are saved and restored across sessions for both log and text views. The command is bound to CTRL+S by default.
• In pager mode (activated by the -q flag), the input will always be written to the terminal if it is smaller than the height. Previously, if the input took awhile to generate, nothing would be written. Also, when the input is larger than the terminal height, any marked lines will be written to the terminal.
• Introducing "Log-Oriented Debugging", a collection of features to streamline mapping log messages back to the source code that generated them. For example, given the log message "Hello, Steve!" and the source directory containing the log statement. lnav can find the line of code that generated the message, such as `logging.info("