[RFC] Daemon to provide structured storage events: peripety

[Gris Ge <fge@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2592 bytes --]

Hi Guys,

In order to provide structured event for storage subsystem.
I created a daemon for append structured data and device ID to
systems logs:

    https://github.com/cathay4t/peripety/

Features:
 * Use sysfs and devfs for information query only. Designed to handle
   log burst.

 * Saves logs in journald with structured data. Log are hide by default
   to eliminate the duplicate entry. Try `journalctl  -o json-pretty
   IS_PERIPETY=TRUE`

 * Provides consistent id for device of the log.

 * Provides extra information like mount point on fs mount, FC/iSCSI
   path information on multipath path failure:
    https://github.com/cathay4t/peripety/blob/master/examples/fs/ext4_mount_lv_mpath_scsi.json
    https://github.com/cathay4t/peripety/blob/master/examples/mpath/mpath_fc_path_offline.json

 * Provides device dependency information. E.g. LV over mpath over SCSI.
   Could query all related logs(fs, lvm, mpath, scsi) of physical device
   by command:

       journalctl DEV_WWID=<WWID> + OWNERS_WWIDS=<WWID>

 * Allows user defined regex in /etc/peripetyd.conf.

 * TBD: C/python/rust lightweight library for query block information
   on all kind of dev string(major:minor, scsi_id, nvme ctrl_id+ns_id,
   etc).

 * TODO: Varlink(json) interface.

 * TODO: Handle user space tool logs like mulitpathd, iscsid.


Example JSON event of multipath FC path failure:
    {
      "hostname": "storageqe-18.rhts.eng.bos.redhat.com",
      "severity": "Warning",
      "sub_system": "Multipath",
      "timestamp": 1526053021631595,
      "event_id": "",
      "event_type": "DM_MPATH_PATH_FAILED",
      "dev_wwid": "360a98000324669436c2b45666c567946",
      "dev_path": "/dev/mapper/360a98000324669436c2b45666c567946",
      "owners_wwids": [
        "t10.NETAPP-LUN-2FiCl+EflVyF"
      ],
      "owners_paths": [
        "/dev/sdd"
      ],
      "kdev": "8:48",
      "msg": "device-mapper: multipath: Failing path 8:48.",
      "extention": {
        "blk_major_minor": "8:48",
        "transport": "FC",
        "port_state": "Online",
        "target_wwpn": "0x500a0982891b8dc5",
        "host_wwpn": "0x10000000c9a02834",
        "speed": "8 Gbit",
        "driver_name": "lpfc"
      }
    }

I have also created some kernel patches to improve this:
 * Provides WWID by kernel on every log to fix the race issue.
 * Provides event type by kernel to save the use space regex captures.
https://github.com/cathay4t/linux/commits/structured_log

Thank you for your time.

Best regards.

-- 
Gris Ge

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]