[-- Attachment #1: Type: text/plain, Size: 413 bytes --] **I am looking for a possible partnership with a business or individual in your country so that I can make some investments there due to the prolonged civil war in my country, now the earthquake in Syria. I wish to relocate my investment capital to your country, any viable investment idea you suggest to me will be considered.* *Kindly, reply for us to discuss this further. Regards* *Al.Nabil, 69 years old.*
Hi,
On 14/06/18 15:43, Richard W.M. Jones wrote:
> [Adding libguestfs mailing list]
>
> On Thu, Jun 14, 2018 at 04:15:57PM +0200, Vojtěch Trefný wrote:
>>
>> On 06/14/2018 01:15 PM, Richard W.M. Jones wrote:
>>> libguestfs provides a C library and large set of tools for
>>> manipulating disk images. http://libguestfs.org/
>>>
>>> As part of this we provide APIs to open VM disks and do things like
>>> enumerate partitions or resize logical volumes. The actual way this
>>> works currently is we run the external commands (eg. parted, lvresize)
>>> inside a small virtual machine and pass the right command line options
>>> or parse the output. In some cases we're also parsing stuff out of
>>> kernel /sys/block.
>>>
>>> We've accumulated a large amount of code to do this (I counted 60619
>>> lines of code in the current version). Here are a few examples so you
>>> can see in concrete terms what I'm talking about:
>>>
>>> https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/parted.c#L344
>>>
>>> https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/lvm.c#L271
>>>
>>> Steven W pointed me to "Project Springfield" and it sort of looks like
>>> it's in the same area. Could libguestfs replace the parsing code
>>> above with this?
>>>
>>> What might be problems: We have no python or dbus in the appliance.
>>> So anything that depends on those is a non-starter.
>>>
>>> TBH the project webpage left me more confused than enlightened. There
>>> seem to be lots of projects (subprojects?) doing stuff with odd names
>>> and no unifying philosophy, and I'm not sure if Project Springfield is
>>> a thing or more of an intention.
>>>
>>> Rich.
>>>
>> Hi, project springfield is a collection of existing projects,
>> libblockdev is probably the "subproject" you are looking for. It is
>> a plugin based C library -- we have plugins for working with btrfs,
>> filesystems (ext, xfs, vfat and ntfs), lvm, mdraid, partitions etc.
>> It mostly also uses the command line tools and in some cases also
>> other existing libraries (libcryptsetup, libmount, libparted...).
>>
>> Libblockdev Github repo: https://github.com/storaged-project/libblockdev
>> and API documentation: http://storaged.org/libblockdev/
>>
>> From the code you sent it looks like libblockdev covers most of the
>> functionality libguestfs needs. Some functionality is missing (e.g.
>> we don't support changing of uuid for lvs) but we can add these
>> missing bits.
>>
>> --
>> Vojtech Trefny
If there is scope for cooperation in this area, then that is very much
what we hope to achieve with Springfield. I'm sure that we'll find that
there are a number of projects that we were not initially aware of that
might be good areas for collaboration/inspiration as we progress. So
that is one reason for the slightly vague description at the moment - I
expect this to tighten up a bit as things mature.
That said we do want to make sure that the docs on the web site are
helpful in explaining what Springfield is all about, so suggestions are
very welcome as to how it could be improved. Are there specific parts
that are confusing?
One reason that there has not yet been the overall unification of
subprojects is that this will take time. Initially there are quite a
large number of "obvious" gaps in the current tooling, so a lot of the
initial work has just been in addressing those gaps. One thing that Paul
Evans has been working on, for example, is to ensure that there is good
filesystem support for PCP, since that is a good way to get a
performance monitoring framework up and running quickly.
Jan Tulak started looking at how we might look to provide a unified
interface to the various filesystem utilities, however that is a tricky
area, since it requires buy-in from the various maintainers of the
utilities and there is a bit of chicken-and-egg in relation to wanting
something to be useful before adopting it. I'm sure we'll get there in
the end, but it will take some time to agree the right approach.
Certainly libguestfs sounds like the kind of project that might be
useful to integrate with Springfield. One way to help figure that out
would be to ask some questions....
- Are there features which libguestfs requires, which have perhaps
already been solved in one of the Springfield subprojects?
- Are there features which Springfield should have, but which are
already solved in libguestfs?
So sharing todo lists/requirements would be a good start in looking for
common ground.
On the technology side, I don't think that there are any hard and fast
rules on what we use in Springfield. My own hope is that it will remain
modular enough that we can easily change components if required - as
with any project, not every decision made at the start will necessarily
be optimal, so my feeling is that we should design with that in mind and
make it easy to replace/upgrade components in the future, so far as
possible.
I hope that gives a bit more background,
Steve.
[Adding libguestfs mailing list] On Thu, Jun 14, 2018 at 04:15:57PM +0200, VojtÄch Trefný wrote: > > > On 06/14/2018 01:15 PM, Richard W.M. Jones wrote: > > > >libguestfs provides a C library and large set of tools for > >manipulating disk images. http://libguestfs.org/ > > > >As part of this we provide APIs to open VM disks and do things like > >enumerate partitions or resize logical volumes. The actual way this > >works currently is we run the external commands (eg. parted, lvresize) > >inside a small virtual machine and pass the right command line options > >or parse the output. In some cases we're also parsing stuff out of > >kernel /sys/block. > > > >We've accumulated a large amount of code to do this (I counted 60619 > >lines of code in the current version). Here are a few examples so you > >can see in concrete terms what I'm talking about: > > > >https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/parted.c#L344 > > > >https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/lvm.c#L271 > > > >Steven W pointed me to "Project Springfield" and it sort of looks like > >it's in the same area. Could libguestfs replace the parsing code > >above with this? > > > >What might be problems: We have no python or dbus in the appliance. > >So anything that depends on those is a non-starter. > > > >TBH the project webpage left me more confused than enlightened. There > >seem to be lots of projects (subprojects?) doing stuff with odd names > >and no unifying philosophy, and I'm not sure if Project Springfield is > >a thing or more of an intention. > > > >Rich. > > > > Hi, project springfield is a collection of existing projects, > libblockdev is probably the "subproject" you are looking for. It is > a plugin based C library -- we have plugins for working with btrfs, > filesystems (ext, xfs, vfat and ntfs), lvm, mdraid, partitions etc. > It mostly also uses the command line tools and in some cases also > other existing libraries (libcryptsetup, libmount, libparted...). > > Libblockdev Github repo: https://github.com/storaged-project/libblockdev > and API documentation: http://storaged.org/libblockdev/ > > From the code you sent it looks like libblockdev covers most of the > functionality libguestfs needs. Some functionality is missing (e.g. > we don't support changing of uuid for lvs) but we can add these > missing bits. > > -- > Vojtech Trefny -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
On 06/14/2018 01:15 PM, Richard W.M. Jones wrote: > > libguestfs provides a C library and large set of tools for > manipulating disk images. http://libguestfs.org/ > > As part of this we provide APIs to open VM disks and do things like > enumerate partitions or resize logical volumes. The actual way this > works currently is we run the external commands (eg. parted, lvresize) > inside a small virtual machine and pass the right command line options > or parse the output. In some cases we're also parsing stuff out of > kernel /sys/block. > > We've accumulated a large amount of code to do this (I counted 60619 > lines of code in the current version). Here are a few examples so you > can see in concrete terms what I'm talking about: > > https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/parted.c#L344 > > https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/lvm.c#L271 > > Steven W pointed me to "Project Springfield" and it sort of looks like > it's in the same area. Could libguestfs replace the parsing code > above with this? > > What might be problems: We have no python or dbus in the appliance. > So anything that depends on those is a non-starter. > > TBH the project webpage left me more confused than enlightened. There > seem to be lots of projects (subprojects?) doing stuff with odd names > and no unifying philosophy, and I'm not sure if Project Springfield is > a thing or more of an intention. > > Rich. > Hi, project springfield is a collection of existing projects, libblockdev is probably the "subproject" you are looking for. It is a plugin based C library -- we have plugins for working with btrfs, filesystems (ext, xfs, vfat and ntfs), lvm, mdraid, partitions etc. It mostly also uses the command line tools and in some cases also other existing libraries (libcryptsetup, libmount, libparted...). Libblockdev Github repo: https://github.com/storaged-project/libblockdev and API documentation: http://storaged.org/libblockdev/ From the code you sent it looks like libblockdev covers most of the functionality libguestfs needs. Some functionality is missing (e.g. we don't support changing of uuid for lvs) but we can add these missing bits. -- Vojtech Trefny
libguestfs provides a C library and large set of tools for manipulating disk images. http://libguestfs.org/ As part of this we provide APIs to open VM disks and do things like enumerate partitions or resize logical volumes. The actual way this works currently is we run the external commands (eg. parted, lvresize) inside a small virtual machine and pass the right command line options or parse the output. In some cases we're also parsing stuff out of kernel /sys/block. We've accumulated a large amount of code to do this (I counted 60619 lines of code in the current version). Here are a few examples so you can see in concrete terms what I'm talking about: https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/parted.c#L344 https://github.com/libguestfs/libguestfs/blob/afd1c70601c51043684a0245ce2f63d71a9cc07a/daemon/lvm.c#L271 Steven W pointed me to "Project Springfield" and it sort of looks like it's in the same area. Could libguestfs replace the parsing code above with this? What might be problems: We have no python or dbus in the appliance. So anything that depends on those is a non-starter. TBH the project webpage left me more confused than enlightened. There seem to be lots of projects (subprojects?) doing stuff with odd names and no unifying philosophy, and I'm not sure if Project Springfield is a thing or more of an intention. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-p2v converts physical machines to virtual machines. Boot with a live CD or over the network (PXE) and turn machines into KVM guests. http://libguestfs.org/virt-v2v
On 05/16/2018 07:52 AM, Gris Ge wrote: > Hi Guys, > > In order to provide structured event for storage subsystem. > I created a daemon for append structured data and device ID to > systems logs: > > https://github.com/cathay4t/peripety/ > > Features: > * Use sysfs and devfs for information query only. Designed to handle > log burst. > > * Saves logs in journald with structured data. Log are hide by default > to eliminate the duplicate entry. Try `journalctl -o json-pretty > IS_PERIPETY=TRUE` > > * Provides consistent id for device of the log. > > * Provides extra information like mount point on fs mount, FC/iSCSI > path information on multipath path failure: > https://github.com/cathay4t/peripety/blob/master/examples/fs/ext4_mount_lv_mpath_scsi.json > https://github.com/cathay4t/peripety/blob/master/examples/mpath/mpath_fc_path_offline.json > > * Provides device dependency information. E.g. LV over mpath over SCSI. > Could query all related logs(fs, lvm, mpath, scsi) of physical device > by command: > > journalctl DEV_WWID=<WWID> + OWNERS_WWIDS=<WWID> > > * Allows user defined regex in /etc/peripetyd.conf. > > * TBD: C/python/rust lightweight library for query block information > on all kind of dev string(major:minor, scsi_id, nvme ctrl_id+ns_id, > etc). > > * TODO: Varlink(json) interface. > > * TODO: Handle user space tool logs like mulitpathd, iscsid. > > > Example JSON event of multipath FC path failure: > { > "hostname": "storageqe-18.rhts.eng.bos.redhat.com", > "severity": "Warning", > "sub_system": "Multipath", > "timestamp": 1526053021631595, > "event_id": "", > "event_type": "DM_MPATH_PATH_FAILED", > "dev_wwid": "360a98000324669436c2b45666c567946", > "dev_path": "/dev/mapper/360a98000324669436c2b45666c567946", > "owners_wwids": [ > "t10.NETAPP-LUN-2FiCl+EflVyF" > ], > "owners_paths": [ > "/dev/sdd" > ], > "kdev": "8:48", > "msg": "device-mapper: multipath: Failing path 8:48.", > "extention": { > "blk_major_minor": "8:48", > "transport": "FC", > "port_state": "Online", > "target_wwpn": "0x500a0982891b8dc5", > "host_wwpn": "0x10000000c9a02834", > "speed": "8 Gbit", > "driver_name": "lpfc" > } > } > > I have also created some kernel patches to improve this: > * Provides WWID by kernel on every log to fix the race issue. > * Provides event type by kernel to save the use space regex captures. > https://github.com/cathay4t/linux/commits/structured_log > > Thank you for your time. > > Best regards. > Thanks! For better integration with common logging, I suggest you develop your own common logging schema with namespace(s). We will be happy to assist you. See https://github.com/ViaQ/elasticsearch-templates/tree/master/namespaces for an example of how we break down different components (kubernetes, systemd, docker, aushape, etc.) into their own namespaces with their own fields. The common data model also requires certain fields at the "top level" to be in a certain format. For example, instead of      "timestamp": 1526053021631595, We use      "timestamp": "2018-05-11T09:37:01.631595-06:00", See also https://github.com/ViaQ/elasticsearch-templates/releases/download/0.0.16/com.redhat.viaq-openshift-operations.asciidoc
[-- Attachment #1: Type: text/plain, Size: 2592 bytes --] Hi Guys, In order to provide structured event for storage subsystem. I created a daemon for append structured data and device ID to systems logs: https://github.com/cathay4t/peripety/ Features: * Use sysfs and devfs for information query only. Designed to handle log burst. * Saves logs in journald with structured data. Log are hide by default to eliminate the duplicate entry. Try `journalctl -o json-pretty IS_PERIPETY=TRUE` * Provides consistent id for device of the log. * Provides extra information like mount point on fs mount, FC/iSCSI path information on multipath path failure: https://github.com/cathay4t/peripety/blob/master/examples/fs/ext4_mount_lv_mpath_scsi.json https://github.com/cathay4t/peripety/blob/master/examples/mpath/mpath_fc_path_offline.json * Provides device dependency information. E.g. LV over mpath over SCSI. Could query all related logs(fs, lvm, mpath, scsi) of physical device by command: journalctl DEV_WWID=<WWID> + OWNERS_WWIDS=<WWID> * Allows user defined regex in /etc/peripetyd.conf. * TBD: C/python/rust lightweight library for query block information on all kind of dev string(major:minor, scsi_id, nvme ctrl_id+ns_id, etc). * TODO: Varlink(json) interface. * TODO: Handle user space tool logs like mulitpathd, iscsid. Example JSON event of multipath FC path failure: { "hostname": "storageqe-18.rhts.eng.bos.redhat.com", "severity": "Warning", "sub_system": "Multipath", "timestamp": 1526053021631595, "event_id": "", "event_type": "DM_MPATH_PATH_FAILED", "dev_wwid": "360a98000324669436c2b45666c567946", "dev_path": "/dev/mapper/360a98000324669436c2b45666c567946", "owners_wwids": [ "t10.NETAPP-LUN-2FiCl+EflVyF" ], "owners_paths": [ "/dev/sdd" ], "kdev": "8:48", "msg": "device-mapper: multipath: Failing path 8:48.", "extention": { "blk_major_minor": "8:48", "transport": "FC", "port_state": "Online", "target_wwpn": "0x500a0982891b8dc5", "host_wwpn": "0x10000000c9a02834", "speed": "8 Gbit", "driver_name": "lpfc" } } I have also created some kernel patches to improve this: * Provides WWID by kernel on every log to fix the race issue. * Provides event type by kernel to save the use space regex captures. https://github.com/cathay4t/linux/commits/structured_log Thank you for your time. Best regards. -- Gris Ge [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --]
On Thu, 2018-03-15 at 11:06 +0100, Jan Tulak wrote:
> Does this work?
Yes :)
Does this work?
test