RE: non-persistent storage?

RE: non-persistent storage?

[Buchbinder, Barry (NIH/NIAID) [E] via cygwin]

Eliot Moss sent the following at Thursday, December 12, 2019 2:19 pm
>On 12/12/2019 7:00 AM, Ulli Horlacher wrote:
>
>> I need to store some data (a few kB) non-persistent.
>> On a real UNIX I would use /var/run, because after a shutdown all its
>> content is lost.
>> But on cygwin /var/run is stored on disk.
>>
>> I cannot use an environment variable, because different processes need
>> to read/write the data.
>>
>> /proc is non-persistent (in respect to a reboot), but It is not a
>> generic storage place.
>>
>> What can I use with cygwin instead?
>>
>> Installing third party software is not an option, it must work with a
>> standard Windows (and cygwin).
>
>I would think of temp directories, such as /tmp. They can be cleaned out
>at will on restart, no?

Bash runs the file ~/.bash_logout when it exits.  One could use that to
clear a temp file out of /tmp.

If the temp file was created by mktemp and the name saved in an
environmental variable, each bash shell could have its own file with not
risk that an instance of bash would erase another instance's file.

That assumes that this is consistent with your need for different
processes to read/write data.

Good luck,

- Barry
  Disclaimer: Statements made herein are not made on behalf of NIAID.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Ulli Horlacher]

On Thu 2019-12-12 (21:59), Buchbinder, Barry (NIH/NIAID) [E] via cygwin wrote:

> If the temp file was created by mktemp and the name saved in an
> environmental variable, each bash shell could have its own file with not
> risk that an instance of bash would erase another instance's file.

I need the opposite: all processes must read and write the same data!
And the data must be gone when system shuts down or even if there is a
power failure.

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<BYAPR09MB29010912EBAE9D8F0196D35E9B550@BYAPR09MB2901.namprd09.prod.outlook.com>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Erik Soderquist]

On Thu, Dec 12, 2019 at 5:08 PM Ulli Horlacher wrote:
>
> On Thu 2019-12-12 (21:59), Buchbinder, Barry (NIH/NIAID) [E] via cygwin wrote:
>
> > If the temp file was created by mktemp and the name saved in an
> > environmental variable, each bash shell could have its own file with not
> > risk that an instance of bash would erase another instance's file.
>
> I need the opposite: all processes must read and write the same data!
> And the data must be gone when system shuts down or even if there is a
> power failure.

This is very hacky, but I believe you can get the effect you want by
having an admin process auto-start at host startup, identify itself,
and then set parts of its own /proc/ process ID tree as world
read/write.  I think this will give you the "destroyed even at power
failure" impermanence you are l,booking for.  I know it is not in the
normal layouts (like /var/run/ would be), but we are working around
limitation s imposed by Windows.

-- Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Erik Soderquist]

On Thu, Dec 12, 2019 at 5:45 PM Erik Soderquist wrote:
> This is very hacky, but I believe you can get the effect you want by
> having an admin process auto-start at host startup, identify itself,
> and then set parts of its own /proc/ process ID tree as world
> read/write.  I think this will give you the "destroyed even at power
> failure" impermanence you are l,booking for.  I know it is not in the
> normal layouts (like /var/run/ would be), but we are working around
> limitation s imposed by Windows.

To "move"to /var/run/, the process could also build symlinks to the
location(s) in /proc/ so the other programs/users have a stable
/var/run/ location and only the auto-started process needs to know or
care about the locations in /proc/

-- Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Ulli Horlacher]

On Thu 2019-12-12 (17:45), Erik Soderquist wrote:

> This is very hacky, but I believe you can get the effect you want by
> having an admin process auto-start at host startup, identify itself,
> and then set parts of its own /proc/ process ID tree as world
> read/write.

It is not possible to write any data to /proc using standard IO read/write
like on a tmpfs file system.


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<CACoZoo1crFaE0ZHBVZaps+F93MyX1c4MN5-5awJ=jFmEAE2vnA@mail.gmail.com>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

RE: non-persistent storage?

[Buchbinder, Barry (NIH/NIAID) [E] via cygwin]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1987 bytes --]

>On Thu, Dec 12, 2019 at 5:08 PM Ulli Horlacher wrote:
>
>>Erik Soderquist sent the following at Thursday, December 12, 2019 5:45 pm
>>
>> On Thu 2019-12-12 (21:59), Buchbinder, Barry (NIH/NIAID) [E] via cygwin wrote:
>>>
>>> If the temp file was created by mktemp and the name saved in an
>>> environmental variable, each bash shell could have its own file with
>>> not risk that an instance of bash would erase another instance's file.
>>
>> I need the opposite: all processes must read and write the same data!
>> And the data must be gone when system shuts down or even if there is a
>> power failure.
>
> This is very hacky, but I believe you can get the effect you want by
> having an admin process auto-start at host startup, identify itself, and
> then set parts of its own /proc/ process ID tree as world read/write.
> I think this will give you the "destroyed even at power failure"
> impermanence you are looking for. I know it is not in the normal
> layouts (like /var/run/ would be), but we are working around limitations
> imposed by Windows.

One could put a script, batch file, or a link thereto in one's startup
folder that will run on Windows' boot.  The script/batch file would be
written to clean out the tmp files.  Google << windows startup folder >>.
If all processes, regardless of whether they start with the same
shell/subshell, will share the same temp file, one could create the file
upon boot using the same script.

Also, one could make the script that creates the temp file (e.g.,
.bashrc) check the file or an environmental variable so that a subshell
doesn't create a new temp file, if that's how you want it.

- Barry
  Disclaimer: Statements made herein are not made on behalf of NIAID.

\0ТÒÐÐ¥\a&ö&ÆVÒ\a&W\x06÷'G3¢\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒ÷\a&ö&ÆV×2æ‡FÖÀФd\x15\x13¢\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöf\x17\x12ðФFö7VÖVçF\x17F–öã¢\x02\x02\x02\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöFö72æ‡FÖÀÐ¥Vç7V'67&–&R\x06–æfó¢\x02\x02\x02\x02\x02\x06‡GG\x03¢òö7–wv–âæ6öÒöÖÂò7Vç7V'67&–&R×6–×\x06ÆPРÐ

Re: non-persistent storage?

[Erik Soderquist]

On Fri, Dec 13, 2019 at 11:20 AM Buchbinder, Barry (NIH/NIAID) [E] via
cygwin <cygwin@cygwin.com> wrote:
> One could put a script, batch file, or a link thereto in one's startup
> folder that will run on Windows' boot.

The OP's issue is that the data must be destroyed instantly even in
the event of a power failure.

Use case scenario:  thief breaks in and steals the host, just yanking
power cords/etc out, and then pulls the drive and mounts it on another
system to recover this data; it is sensitive enough that it must be
destroyed immediately by the power failure.

Unfortunately, nothing in Windows is designed for that kind of
security, and Cygwin has to work around a lot of Windows design flaws
to function in general.

I've test all of the suggestions I've seen so far with the exception
of the cygserver and shared memory, and all of the ones I've tested
failed the power failure scenario.  I think if the cygserver/shared
memory suggestion works, it will be the only available option to
prevent the data from default existing on the disc, and due to another
of Window's design flaws, may still be written into the page file,
though it would be much harder to cleanly extract from the page file
than from a normal filesystem file.

-- Erik

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Steven Penny]

On Fri, 13 Dec 2019 17:57:23 -0500, Erik Soderquist wrote:
> I've test all of the suggestions I've seen so far with the exception
> of the cygserver and shared memory, and all of the ones I've tested
> failed the power failure scenario.

/dev/clipboard?


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Jürgen Wagner]

[-- Attachment #1: Type: text/plain, Size: 938 bytes --]

I can see two options:
1. You create a RAM disk and the contents will obviously be gone when 
the system reboots or crashes.
2. You encrypt the data with a ephemeral key that will be lost on reboot 
(e.g., kept in shared memory). When the system comes up and finds itself 
unable to read the data, it will erase it.

That doesn't really have much to do with Cygwin or Windows - you can do 
that on any system.

On 12.12.2019 23:07, Ulli Horlacher wrote:
> On Thu 2019-12-12 (21:59), Buchbinder, Barry (NIH/NIAID) [E] via cygwin wrote:
>
>> If the temp file was created by mktemp and the name saved in an
>> environmental variable, each bash shell could have its own file with not
>> risk that an instance of bash would erase another instance's file.
> I need the opposite: all processes must read and write the same data!
> And the data must be gone when system shuts down or even if there is a
> power failure.
>



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3469 bytes --]

Re: non-persistent storage?

[Ulli Horlacher]

On Fri 2019-12-13 (00:03), Jürgen Wagner wrote:

> 1. You create a RAM disk

How do I do it with cygwin?


> 2. You encrypt the data with a ephemeral key

This is exactly what I want to share between processes!


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<3bcb25a9-3d7a-2b88-f2ca-83d46f2b03cd@wagner.is>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Jürgen Wagner]

[-- Attachment #1: Type: text/plain, Size: 750 bytes --]

Ok, we're getting closer.

- Run cygserver-config as an administrator.
- Do "net start cygserver"

That will provide shared memory capabilities. Then try this:

# shmtool w myFunnyC8380ufsKey
Shared memory segment exists - opening as client
Done...

# shmtool r
Shared memory segment exists - opening as client
segptr: myFunnyC8380ufsKey

#

On reboot of the host Windows, the shared memory segment will not be 
persisted.

Cheers,
--j.




On 13.12.2019 00:21, Ulli Horlacher wrote:
> On Fri 2019-12-13 (00:03), Jürgen Wagner wrote:
>
>> 1. You create a RAM disk
> How do I do it with cygwin?
>
>
>> 2. You encrypt the data with a ephemeral key
> This is exactly what I want to share between processes!
>
>



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3469 bytes --]

Re: non-persistent storage?

[Ulli Horlacher]

On Fri 2019-12-13 (00:51), Jürgen Wagner wrote:

> - Run cygserver-config as an administrator.

~: cygserver-config
Generating /etc/cygserver.conf file

Warning: The following function requires administrator privileges!

Do you want to install cygserver as service?
(Say "no" if it's already installed as service) (yes/no) yes
/usr/bin/cygserver-config: line 181: cygrunsrv: command not found


> # shmtool w myFunnyC8380ufsKey

shmtool is also missing.
Which package have I to install?


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<72eb3b54-1c86-689e-f663-543f80755239@wagner.is>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Marco Atzeri]

Am 13.12.2019 um 11:06 schrieb Ulli Horlacher:
> On Fri 2019-12-13 (00:51), Jürgen Wagner wrote:
> 
>> - Run cygserver-config as an administrator.
> 
> ~: cygserver-config
> Generating /etc/cygserver.conf file
> 
> Warning: The following function requires administrator privileges!
> 
> Do you want to install cygserver as service?
> (Say "no" if it's already installed as service) (yes/no) yes
> /usr/bin/cygserver-config: line 181: cygrunsrv: command not found
> 
> 
>> # shmtool w myFunnyC8380ufsKey
> 
> shmtool is also missing.
> Which package have I to install?
> 
> 

$ cygcheck -p shmtool
Found 6 matches for shmtool
...
cygutils-extra-1.4.16-2 - cygutils-extra: A collection of simple 
utilities (other tools)

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Mark Geisert]

Ulli Horlacher wrote:
> On Fri 2019-12-13 (00:51), Jürgen Wagner wrote:
> 
>> - Run cygserver-config as an administrator.
> 
> ~: cygserver-config
> Generating /etc/cygserver.conf file
> 
> Warning: The following function requires administrator privileges!
> 
> Do you want to install cygserver as service?
> (Say "no" if it's already installed as service) (yes/no) yes
> /usr/bin/cygserver-config: line 181: cygrunsrv: command not found

In addition to Marco's reply for shmtool, you'll need the 'cygrunsrv' package.

..mark

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Jürgen Wagner]

[-- Attachment #1: Type: text/plain, Size: 542 bytes --]

And regarding the tmpfs (RAM disk) , you can use /dev/shm.

# date > /dev/shm/key
# cat /dev/shm/key
Fri Dec 13 07:26:03 CET 2019
#

It is not advisable to store larger amounts of data there, but an 
encryption key should be fine.

Cheers,
--j.

On 13.12.2019 00:21, Ulli Horlacher wrote:
> On Fri 2019-12-13 (00:03), Jürgen Wagner wrote:
>
>> 1. You create a RAM disk
> How do I do it with cygwin?
>
>
>> 2. You encrypt the data with a ephemeral key
> This is exactly what I want to share between processes!
>
>



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3469 bytes --]

Re: non-persistent storage?

[Ulli Horlacher]

On Fri 2019-12-13 (07:27), Jürgen Wagner wrote:
> And regarding the tmpfs (RAM disk) , you can use /dev/shm.
> 
> # date > /dev/shm/key
> # cat /dev/shm/key
> Fri Dec 13 07:26:03 CET 2019

/dev/shm ist not a tmpfs on cygwin:

/dev/shm: df -TH /dev/shm
Filesystem     Type  Size  Used Avail Use% Mounted on
C:/cygwin64    ntfs   34G   25G  9.1G  74% /

When I write something into this direcory and reboot, it is still there.


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum TIK         
Universitaet Stuttgart         E-Mail: horlacher@tik.uni-stuttgart.de
Allmandring 30a                Tel:    ++49-711-68565868
70569 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/
REF:<019cfbcf-d1b3-ddf0-1f07-5c2fda599144@wagner.is>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 652 bytes --]

On Dec 13 10:34, Ulli Horlacher wrote:
> On Fri 2019-12-13 (07:27), Jürgen Wagner wrote:
> > And regarding the tmpfs (RAM disk) , you can use /dev/shm.
> > 
> > # date > /dev/shm/key
> > # cat /dev/shm/key
> > Fri Dec 13 07:26:03 CET 2019
> 
> /dev/shm ist not a tmpfs on cygwin:
> 
> /dev/shm: df -TH /dev/shm
> Filesystem     Type  Size  Used Avail Use% Mounted on
> C:/cygwin64    ntfs   34G   25G  9.1G  74% /
> 
> When I write something into this direcory and reboot, it is still there.

...unless you open the file with the O_TMPFILE flag as outlined
in my other reply.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: non-persistent storage?

[Andrey Repin]

Greetings, Ulli Horlacher!

> /dev/shm ist not a tmpfs on cygwin:

Cygwin is a userspace DLL. It is not a filesystem driver.

> /dev/shm: df -TH /dev/shm
> Filesystem     Type  Size  Used Avail Use% Mounted on
> C:/cygwin64    ntfs   34G   25G  9.1G  74% /

> When I write something into this direcory and reboot, it is still there.

Use iSCSI RAM disk?
http://woshub.com/create-ram-disk-windows-server/


-- 
With best regards,
Andrey Repin
Tuesday, December 17, 2019 9:29:29

Sorry for my terrible english...


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: non-persistent storage?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 598 bytes --]

On Dec 17 09:32, Andrey Repin wrote:
> Greetings, Ulli Horlacher!
> 
> > /dev/shm ist not a tmpfs on cygwin:
> 
> Cygwin is a userspace DLL. It is not a filesystem driver.
> 
> > /dev/shm: df -TH /dev/shm
> > Filesystem     Type  Size  Used Avail Use% Mounted on
> > C:/cygwin64    ntfs   34G   25G  9.1G  74% /
> 
> > When I write something into this direcory and reboot, it is still there.
> 
> Use iSCSI RAM disk?
> http://woshub.com/create-ram-disk-windows-server/

Great idea!  Pity this only works on server machines.


Corinna

-- 
Corinna Vinschen
Cygwin Maintainer

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]