* [PATCH rebase] Add support for Compact OS compression for Cygwin
@ 2022-07-14 10:02 Christian Franke
2022-07-14 10:24 ` Corinna Vinschen
0 siblings, 1 reply; 6+ messages in thread
From: Christian Franke @ 2022-07-14 10:02 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 188 bytes --]
[Sorry if this is the wrong list]
This finally completes '--compact-os' support of Cygwin setup.
https://sourceware.org/pipermail/cygwin-apps/2021-May/041225.html
--
Regards
Christian
[-- Attachment #2: 0001-Add-support-for-Compact-OS-compression-for-Cygwin.patch --]
[-- Type: text/plain, Size: 10214 bytes --]
From 807ae9fbaef18491f3aa1e94e66dd21eb6748c3e Mon Sep 17 00:00:00 2001
From: Christian Franke <christian.franke@t-online.de>
Date: Thu, 14 Jul 2022 11:59:50 +0200
Subject: [PATCH] Add support for Compact OS compression for Cygwin
Preserve compression of manually rebased files.
Align compression with Cygwin DLL if database is used.
Only check for writability if file needs rebasing to keep
compression of unchanged files.
Signed-off-by: Christian Franke <christian.franke@t-online.de>
---
rebase.c | 199 +++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 155 insertions(+), 44 deletions(-)
diff --git a/rebase.c b/rebase.c
index a403c85..06828bb 100644
--- a/rebase.c
+++ b/rebase.c
@@ -39,6 +39,10 @@
#include <errno.h>
#include "imagehelper.h"
#include "rebase-db.h"
+#if defined(__CYGWIN__)
+#include <io.h>
+#include <versionhelpers.h>
+#endif
BOOL save_image_info ();
BOOL load_image_info ();
@@ -48,6 +52,10 @@ void print_image_info ();
BOOL rebase (const char *pathname, ULONG64 *new_image_base, BOOL down_flag);
void parse_args (int argc, char *argv[]);
unsigned long long string_to_ulonglong (const char *string);
+#if defined(__CYGWIN__)
+static int compactos_get_algorithm (const char *pathname);
+static int compactos_compress_file (const char *pathname, int algorithm);
+#endif
void usage ();
void help ();
BOOL is_rebaseable (const char *pathname);
@@ -259,9 +267,19 @@ main (int argc, char *argv[])
ULONG64 new_image_base = image_base;
for (i = 0; i < img_info_size; ++i)
{
+#if defined(__CYGWIN__)
+ int compactos_algorithm
+ = compactos_get_algorithm (img_info_list[i].name);
+#endif
status = rebase (img_info_list[i].name, &new_image_base, down_flag);
if (!status)
return 2;
+#if defined(__CYGWIN__)
+ /* Reapply previous compression. */
+ if (compactos_algorithm >= 0)
+ compactos_compress_file (img_info_list[i].name,
+ compactos_algorithm);
+#endif
}
}
else
@@ -269,6 +287,9 @@ main (int argc, char *argv[])
/* Rebase with database support. */
BOOL header;
+#if defined(__CYGWIN__)
+ int compactos_algorithm = compactos_get_algorithm ("/bin/cygwin1.dll");
+#endif
if (merge_image_info () < 0)
return 2;
status = TRUE;
@@ -279,6 +300,14 @@ main (int argc, char *argv[])
status = rebase (img_info_list[i].name, &new_image_base, FALSE);
if (status)
img_info_list[i].flag.needs_rebasing = 0;
+#if defined(__CYGWIN__)
+ /* If Cygwin DLL is compressed, assume setup was used with option
+ --compact-os. Align compression with Cygwin DLL. */
+ if (compactos_algorithm >= 0
+ && compactos_compress_file (img_info_list[i].name,
+ compactos_algorithm) < 0)
+ compactos_algorithm = -1;
+#endif
}
for (header = FALSE, i = 0; i < img_info_size; ++i)
if (img_info_list[i].flag.cannot_rebase == 1)
@@ -589,6 +618,7 @@ set_cannot_rebase (img_info_t *img)
* is set to 2 on loading the database entries */
if (img->flag.cannot_rebase <= 1 )
{
+ /* This also removes Compact OS compression. */
int fd = open (img->name, O_WRONLY);
if (fd < 0)
img->flag.cannot_rebase = 1;
@@ -711,7 +741,7 @@ merge_image_info ()
to match with reality. */
for (i = 0; i < img_info_rebase_start; ++i)
{
- ULONG64 cur_base;
+ ULONG64 cur_base, cur_base_orig;
ULONG cur_size, slot_size;
/* Files with the needs_rebasing or cannot_rebase flags set have been
@@ -733,55 +763,61 @@ merge_image_info ()
continue;
}
slot_size = roundup2 (cur_size, ALLOCATION_SLOT);
- if (set_cannot_rebase (&img_info_list[i]))
- img_info_list[i].base = cur_base;
- else
+ cur_base_orig = cur_base;
+ /* If the file has been reinstalled, try to rebase to the same address
+ in the first place. */
+ if (cur_base != img_info_list[i].base)
{
- /* If the file has been reinstalled, try to rebase to the same address
- in the first place. */
- if (cur_base != img_info_list[i].base)
- {
- img_info_list[i].flag.needs_rebasing = 1;
- if (verbose)
- fprintf (stderr, "rebasing %s because it's base has changed (due to being reinstalled?)\n", img_info_list[i].name);
- /* Set cur_base to the old base to simplify subsequent tests. */
- cur_base = img_info_list[i].base;
- }
- /* However, if the DLL got bigger and doesn't fit into its slot
- anymore, rebase this DLL from scratch. */
- if (i + 1 < img_info_rebase_start
- && cur_base + slot_size + offset > img_info_list[i + 1].base)
- {
- img_info_list[i].base = 0;
- if (verbose)
- fprintf (stderr, "rebasing %s because it won't fit in it's old slot without overlapping next DLL\n", img_info_list[i].name);
- }
- /* Does the previous DLL reach into the address space of this
- DLL? This happens if the previous DLL is not rebaseable. */
- else if (i > 0 && cur_base < img_info_list[i - 1].base
- + img_info_list[i - 1].slot_size)
- {
- img_info_list[i].base = 0;
- if (verbose)
- fprintf (stderr, "rebasing %s because previous DLL now overlaps\n", img_info_list[i].name);
- }
- /* Does the file match the base address requirements? If not,
- rebase from scratch. */
- else if ((down_flag && cur_base + slot_size + offset > image_base)
- || (!down_flag && cur_base < image_base))
- {
- img_info_list[i].base = 0;
- if (verbose)
- fprintf (stderr, "rebasing %s because it's base address is outside the expected area\n", img_info_list[i].name);
- }
+ img_info_list[i].flag.needs_rebasing = 1;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because it's base has changed (due to being reinstalled?)\n", img_info_list[i].name);
+ /* Set cur_base to the old base to simplify subsequent tests. */
+ cur_base = img_info_list[i].base;
+ }
+ /* However, if the DLL got bigger and doesn't fit into its slot
+ anymore, rebase this DLL from scratch. */
+ if (i + 1 < img_info_rebase_start
+ && cur_base + slot_size + offset > img_info_list[i + 1].base)
+ {
+ img_info_list[i].base = 0;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because it won't fit in it's old slot without overlapping next DLL\n", img_info_list[i].name);
+ }
+ /* Does the previous DLL reach into the address space of this
+ DLL? This happens if the previous DLL is not rebaseable. */
+ else if (i > 0 && cur_base < img_info_list[i - 1].base
+ + img_info_list[i - 1].slot_size)
+ {
+ img_info_list[i].base = 0;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because previous DLL now overlaps\n", img_info_list[i].name);
+ }
+ /* Does the file match the base address requirements? If not,
+ rebase from scratch. */
+ else if ((down_flag && cur_base + slot_size + offset > image_base)
+ || (!down_flag && cur_base < image_base))
+ {
+ img_info_list[i].base = 0;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because it's base address is outside the expected area\n", img_info_list[i].name);
}
- /* Unconditionally overwrite old with new size. */
- img_info_list[i].size = cur_size;
- img_info_list[i].slot_size = slot_size;
/* Make sure all DLLs with base address 0 have the needs_rebasing
flag set. */
if (img_info_list[i].base == 0)
img_info_list[i].flag.needs_rebasing = 1;
+ /* Only check for writability if file needs rebasing to keep
+ Compact OS compression of unchanged files. Revert rebase
+ decision if not writeable. */
+ if (img_info_list[i].flag.needs_rebasing && set_cannot_rebase (&img_info_list[i]))
+ {
+ img_info_list[i].flag.needs_rebasing = 0;
+ img_info_list[i].base = cur_base_orig;
+ if (verbose)
+ fprintf (stderr, "rebasing %s deferred because file is not writable\n", img_info_list[i].name);
+ }
+ /* Unconditionally overwrite old with new size. */
+ img_info_list[i].size = cur_size;
+ img_info_list[i].slot_size = slot_size;
}
/* The remainder of the function expects img_info_size to be > 0. */
if (img_info_size == 0)
@@ -1380,6 +1416,81 @@ string_to_ulonglong (const char *string)
return number;
}
+#if defined(__CYGWIN__)
+static int
+compactos_get_algorithm (const char *pathname)
+{
+ /* Requires Win10. */
+ if (!IsWindows10OrGreater ())
+ return -1;
+
+ int fd = open (pathname, O_RDONLY);
+ if (fd == -1)
+ return -1;
+
+ struct {
+ WOF_EXTERNAL_INFO Wof;
+ FILE_PROVIDER_EXTERNAL_INFO_V1 FileProvider;
+ } wfp = { {0, 0}, {0, 0, 0} };
+
+ int rc;
+ if (!DeviceIoControl ((HANDLE) _get_osfhandle (fd),
+ FSCTL_GET_EXTERNAL_BACKING, NULL, 0,
+ &wfp, sizeof(wfp), NULL, NULL))
+ rc = -1;
+ else
+ rc = (wfp.Wof.Provider == WOF_PROVIDER_FILE ?
+ wfp.FileProvider.Algorithm : -1);
+
+ close (fd);
+ return rc;
+}
+
+static int
+compactos_compress_file (const char *pathname, int algorithm)
+{
+ /* Do not apply unknown algorithms. */
+ if (!(FILE_PROVIDER_COMPRESSION_XPRESS4K <= algorithm &&
+ algorithm <= FILE_PROVIDER_COMPRESSION_XPRESS16K))
+ return 0;
+
+ int fd = open (pathname, O_RDONLY);
+ if (fd == -1)
+ return -1;
+ HANDLE h = (HANDLE) _get_osfhandle (fd);
+
+ /* Older versions of Win10 set mtime to current time. */
+ FILETIME ft;
+ BOOL ft_valid = GetFileTime (h, NULL, NULL, &ft);
+
+ struct {
+ WOF_EXTERNAL_INFO Wof;
+ FILE_PROVIDER_EXTERNAL_INFO_V1 FileProvider;
+ } wfp;
+ wfp.Wof.Version = WOF_CURRENT_VERSION;
+ wfp.Wof.Provider = WOF_PROVIDER_FILE;
+ wfp.FileProvider.Version = FILE_PROVIDER_CURRENT_VERSION;
+ wfp.FileProvider.Algorithm = algorithm;
+ wfp.FileProvider.Flags = 0;
+
+ int rc;
+ if (!DeviceIoControl (h, FSCTL_SET_EXTERNAL_BACKING, &wfp, sizeof(wfp),
+ NULL, 0, NULL, NULL))
+ rc = (GetLastError() == ERROR_COMPRESSION_NOT_BENEFICIAL ? 0 : -1);
+ else
+ rc = 1;
+
+ if (ft_valid)
+ SetFileTime (h, NULL, NULL, &ft);
+
+ close (fd);
+ if (verbose)
+ printf ("%s: Compact OS algorithm %d %s\n", pathname, algorithm,
+ (rc < 0 ? "FAILED" : rc == 0 ? "not applied " : "applied"));
+ return rc;
+}
+#endif
+
void
usage ()
{
--
2.37.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH rebase] Add support for Compact OS compression for Cygwin
2022-07-14 10:02 [PATCH rebase] Add support for Compact OS compression for Cygwin Christian Franke
@ 2022-07-14 10:24 ` Corinna Vinschen
2022-07-14 12:12 ` Christian Franke
0 siblings, 1 reply; 6+ messages in thread
From: Corinna Vinschen @ 2022-07-14 10:24 UTC (permalink / raw)
To: cygwin-patches
On Jul 14 12:02, Christian Franke wrote:
> [Sorry if this is the wrong list]
Yes, in theorie, but no worries. However...
> This finally completes '--compact-os' support of Cygwin setup.
> https://sourceware.org/pipermail/cygwin-apps/2021-May/041225.html
>
> --
> Regards
> Christian
>
> From 807ae9fbaef18491f3aa1e94e66dd21eb6748c3e Mon Sep 17 00:00:00 2001
> From: Christian Franke <christian.franke@t-online.de>
> Date: Thu, 14 Jul 2022 11:59:50 +0200
> Subject: [PATCH] Add support for Compact OS compression for Cygwin
>
> Preserve compression of manually rebased files.
> Align compression with Cygwin DLL if database is used.
> Only check for writability if file needs rebasing to keep
> compression of unchanged files.
>
> Signed-off-by: Christian Franke <christian.franke@t-online.de>
> ---
> rebase.c | 199 +++++++++++++++++++++++++++++++++++++++++++------------
> 1 file changed, 155 insertions(+), 44 deletions(-)
>
> diff --git a/rebase.c b/rebase.c
> index a403c85..06828bb 100644
> --- a/rebase.c
> +++ b/rebase.c
> @@ -39,6 +39,10 @@
> #include <errno.h>
> #include "imagehelper.h"
> #include "rebase-db.h"
> +#if defined(__CYGWIN__)
> +#include <io.h>
> +#include <versionhelpers.h>
> +#endif
>
> BOOL save_image_info ();
> BOOL load_image_info ();
> @@ -48,6 +52,10 @@ void print_image_info ();
> BOOL rebase (const char *pathname, ULONG64 *new_image_base, BOOL down_flag);
> void parse_args (int argc, char *argv[]);
> unsigned long long string_to_ulonglong (const char *string);
> +#if defined(__CYGWIN__)
> +static int compactos_get_algorithm (const char *pathname);
> +static int compactos_compress_file (const char *pathname, int algorithm);
> +#endif
> void usage ();
> void help ();
> BOOL is_rebaseable (const char *pathname);
> @@ -259,9 +267,19 @@ main (int argc, char *argv[])
> ULONG64 new_image_base = image_base;
> for (i = 0; i < img_info_size; ++i)
> {
> +#if defined(__CYGWIN__)
Given compactos stuff is a OS thingy and not actually a Cygwin feature,
why do we need an ifdef CYGWIN?
> + int compactos_algorithm
> + = compactos_get_algorithm (img_info_list[i].name);
> +#endif
> status = rebase (img_info_list[i].name, &new_image_base, down_flag);
> if (!status)
> return 2;
> +#if defined(__CYGWIN__)
> + /* Reapply previous compression. */
> + if (compactos_algorithm >= 0)
> + compactos_compress_file (img_info_list[i].name,
> + compactos_algorithm);
> +#endif
> }
> }
> else
> @@ -269,6 +287,9 @@ main (int argc, char *argv[])
> /* Rebase with database support. */
> BOOL header;
>
> +#if defined(__CYGWIN__)
> + int compactos_algorithm = compactos_get_algorithm ("/bin/cygwin1.dll");
> +#endif
> if (merge_image_info () < 0)
> return 2;
> status = TRUE;
> @@ -279,6 +300,14 @@ main (int argc, char *argv[])
> status = rebase (img_info_list[i].name, &new_image_base, FALSE);
> if (status)
> img_info_list[i].flag.needs_rebasing = 0;
> +#if defined(__CYGWIN__)
> + /* If Cygwin DLL is compressed, assume setup was used with option
> + --compact-os. Align compression with Cygwin DLL. */
> + if (compactos_algorithm >= 0
> + && compactos_compress_file (img_info_list[i].name,
> + compactos_algorithm) < 0)
> + compactos_algorithm = -1;
> +#endif
This ifdef still makes sense, of course and on first glance, the
remainder of the patch LGTM.
Thanks,
Corinna
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH rebase] Add support for Compact OS compression for Cygwin
2022-07-14 10:24 ` Corinna Vinschen
@ 2022-07-14 12:12 ` Christian Franke
2022-07-14 14:28 ` Corinna Vinschen
0 siblings, 1 reply; 6+ messages in thread
From: Christian Franke @ 2022-07-14 12:12 UTC (permalink / raw)
To: cygwin-patches
[-- Attachment #1: Type: text/plain, Size: 3832 bytes --]
Corinna Vinschen wrote:
> On Jul 14 12:02, Christian Franke wrote:
>> [Sorry if this is the wrong list]
> Yes, in theorie, but no worries. However...
What is the correct list in theory ?-)
>
>> From 807ae9fbaef18491f3aa1e94e66dd21eb6748c3e Mon Sep 17 00:00:00 2001
>> From: Christian Franke <christian.franke@t-online.de>
>> Date: Thu, 14 Jul 2022 11:59:50 +0200
>> Subject: [PATCH] Add support for Compact OS compression for Cygwin
>>
>> Preserve compression of manually rebased files.
>> Align compression with Cygwin DLL if database is used.
>> Only check for writability if file needs rebasing to keep
>> compression of unchanged files.
>>
>> Signed-off-by: Christian Franke <christian.franke@t-online.de>
>> ---
>> rebase.c | 199 +++++++++++++++++++++++++++++++++++++++++++------------
>> 1 file changed, 155 insertions(+), 44 deletions(-)
>>
>> diff --git a/rebase.c b/rebase.c
>> index a403c85..06828bb 100644
>> --- a/rebase.c
>> +++ b/rebase.c
>> @@ -39,6 +39,10 @@
>> #include <errno.h>
>> #include "imagehelper.h"
>> #include "rebase-db.h"
>> +#if defined(__CYGWIN__)
>> +#include <io.h>
>> +#include <versionhelpers.h>
>> +#endif
>>
>> BOOL save_image_info ();
>> BOOL load_image_info ();
>> @@ -48,6 +52,10 @@ void print_image_info ();
>> BOOL rebase (const char *pathname, ULONG64 *new_image_base, BOOL down_flag);
>> void parse_args (int argc, char *argv[]);
>> unsigned long long string_to_ulonglong (const char *string);
>> +#if defined(__CYGWIN__)
>> +static int compactos_get_algorithm (const char *pathname);
>> +static int compactos_compress_file (const char *pathname, int algorithm);
>> +#endif
>> void usage ();
>> void help ();
>> BOOL is_rebaseable (const char *pathname);
>> @@ -259,9 +267,19 @@ main (int argc, char *argv[])
>> ULONG64 new_image_base = image_base;
>> for (i = 0; i < img_info_size; ++i)
>> {
>> +#if defined(__CYGWIN__)
> Given compactos stuff is a OS thingy and not actually a Cygwin feature,
> why do we need an ifdef CYGWIN?
Mainly because I didn't test on MSYS and other (which ever these are)
environments. This also requires a recent release of MinGW-w64 headers
(>=10.0.0) which includes (my) Compact OS patch.
>
>> + int compactos_algorithm
>> + = compactos_get_algorithm (img_info_list[i].name);
>> +#endif
>> status = rebase (img_info_list[i].name, &new_image_base, down_flag);
>> if (!status)
>> return 2;
>> +#if defined(__CYGWIN__)
>> + /* Reapply previous compression. */
>> + if (compactos_algorithm >= 0)
>> + compactos_compress_file (img_info_list[i].name,
>> + compactos_algorithm);
>> +#endif
>> }
>> }
>> else
>> @@ -269,6 +287,9 @@ main (int argc, char *argv[])
>> /* Rebase with database support. */
>> BOOL header;
>>
>> +#if defined(__CYGWIN__)
>> + int compactos_algorithm = compactos_get_algorithm ("/bin/cygwin1.dll");
>> +#endif
>> if (merge_image_info () < 0)
>> return 2;
>> status = TRUE;
>> @@ -279,6 +300,14 @@ main (int argc, char *argv[])
>> status = rebase (img_info_list[i].name, &new_image_base, FALSE);
>> if (status)
>> img_info_list[i].flag.needs_rebasing = 0;
>> +#if defined(__CYGWIN__)
>> + /* If Cygwin DLL is compressed, assume setup was used with option
>> + --compact-os. Align compression with Cygwin DLL. */
>> + if (compactos_algorithm >= 0
>> + && compactos_compress_file (img_info_list[i].name,
>> + compactos_algorithm) < 0)
>> + compactos_algorithm = -1;
>> +#endif
> This ifdef still makes sense, of course ...
Could possibly also be enhanced to __MSYS__ and msys1.dll.
> ... and on first glance, the
> remainder of the patch LGTM.
Thanks. Attached is an alternative patch with most ifdefs removed.
Thanks,
Christian
[-- Attachment #2: 0001-Add-support-for-Compact-OS-compression.patch --]
[-- Type: text/plain, Size: 10106 bytes --]
From 5d274510b3caafe68929014e02e351b111569cf7 Mon Sep 17 00:00:00 2001
From: Christian Franke <christian.franke@t-online.de>
Date: Thu, 14 Jul 2022 13:54:12 +0200
Subject: [PATCH] Add support for Compact OS compression
Preserve compression of manually rebased files.
Align compression with Cygwin DLL if database is used.
Only check for writability if file needs rebasing to keep
compression of unchanged files.
Signed-off-by: Christian Franke <christian.franke@t-online.de>
---
rebase.c | 189 ++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 145 insertions(+), 44 deletions(-)
diff --git a/rebase.c b/rebase.c
index a403c85..5cda123 100644
--- a/rebase.c
+++ b/rebase.c
@@ -37,8 +37,10 @@
#include <getopt.h>
#include <string.h>
#include <errno.h>
+#include <io.h>
#include "imagehelper.h"
#include "rebase-db.h"
+#include <versionhelpers.h> /* requires <windows.h> */
BOOL save_image_info ();
BOOL load_image_info ();
@@ -48,6 +50,8 @@ void print_image_info ();
BOOL rebase (const char *pathname, ULONG64 *new_image_base, BOOL down_flag);
void parse_args (int argc, char *argv[]);
unsigned long long string_to_ulonglong (const char *string);
+static int compactos_get_algorithm (const char *pathname);
+static int compactos_compress_file (const char *pathname, int algorithm);
void usage ();
void help ();
BOOL is_rebaseable (const char *pathname);
@@ -259,9 +263,15 @@ main (int argc, char *argv[])
ULONG64 new_image_base = image_base;
for (i = 0; i < img_info_size; ++i)
{
+ int compactos_algorithm
+ = compactos_get_algorithm (img_info_list[i].name);
status = rebase (img_info_list[i].name, &new_image_base, down_flag);
if (!status)
return 2;
+ /* Reapply previous compression. */
+ if (compactos_algorithm >= 0)
+ compactos_compress_file (img_info_list[i].name,
+ compactos_algorithm);
}
}
else
@@ -269,6 +279,9 @@ main (int argc, char *argv[])
/* Rebase with database support. */
BOOL header;
+#if defined(__CYGWIN__)
+ int compactos_algorithm = compactos_get_algorithm ("/bin/cygwin1.dll");
+#endif
if (merge_image_info () < 0)
return 2;
status = TRUE;
@@ -279,6 +292,14 @@ main (int argc, char *argv[])
status = rebase (img_info_list[i].name, &new_image_base, FALSE);
if (status)
img_info_list[i].flag.needs_rebasing = 0;
+#if defined(__CYGWIN__)
+ /* If Cygwin DLL is compressed, assume setup was used with option
+ --compact-os. Align compression with Cygwin DLL. */
+ if (compactos_algorithm >= 0
+ && compactos_compress_file (img_info_list[i].name,
+ compactos_algorithm) < 0)
+ compactos_algorithm = -1;
+#endif
}
for (header = FALSE, i = 0; i < img_info_size; ++i)
if (img_info_list[i].flag.cannot_rebase == 1)
@@ -589,6 +610,7 @@ set_cannot_rebase (img_info_t *img)
* is set to 2 on loading the database entries */
if (img->flag.cannot_rebase <= 1 )
{
+ /* This also removes Compact OS compression. */
int fd = open (img->name, O_WRONLY);
if (fd < 0)
img->flag.cannot_rebase = 1;
@@ -711,7 +733,7 @@ merge_image_info ()
to match with reality. */
for (i = 0; i < img_info_rebase_start; ++i)
{
- ULONG64 cur_base;
+ ULONG64 cur_base, cur_base_orig;
ULONG cur_size, slot_size;
/* Files with the needs_rebasing or cannot_rebase flags set have been
@@ -733,55 +755,61 @@ merge_image_info ()
continue;
}
slot_size = roundup2 (cur_size, ALLOCATION_SLOT);
- if (set_cannot_rebase (&img_info_list[i]))
- img_info_list[i].base = cur_base;
- else
+ cur_base_orig = cur_base;
+ /* If the file has been reinstalled, try to rebase to the same address
+ in the first place. */
+ if (cur_base != img_info_list[i].base)
{
- /* If the file has been reinstalled, try to rebase to the same address
- in the first place. */
- if (cur_base != img_info_list[i].base)
- {
- img_info_list[i].flag.needs_rebasing = 1;
- if (verbose)
- fprintf (stderr, "rebasing %s because it's base has changed (due to being reinstalled?)\n", img_info_list[i].name);
- /* Set cur_base to the old base to simplify subsequent tests. */
- cur_base = img_info_list[i].base;
- }
- /* However, if the DLL got bigger and doesn't fit into its slot
- anymore, rebase this DLL from scratch. */
- if (i + 1 < img_info_rebase_start
- && cur_base + slot_size + offset > img_info_list[i + 1].base)
- {
- img_info_list[i].base = 0;
- if (verbose)
- fprintf (stderr, "rebasing %s because it won't fit in it's old slot without overlapping next DLL\n", img_info_list[i].name);
- }
- /* Does the previous DLL reach into the address space of this
- DLL? This happens if the previous DLL is not rebaseable. */
- else if (i > 0 && cur_base < img_info_list[i - 1].base
- + img_info_list[i - 1].slot_size)
- {
- img_info_list[i].base = 0;
- if (verbose)
- fprintf (stderr, "rebasing %s because previous DLL now overlaps\n", img_info_list[i].name);
- }
- /* Does the file match the base address requirements? If not,
- rebase from scratch. */
- else if ((down_flag && cur_base + slot_size + offset > image_base)
- || (!down_flag && cur_base < image_base))
- {
- img_info_list[i].base = 0;
- if (verbose)
- fprintf (stderr, "rebasing %s because it's base address is outside the expected area\n", img_info_list[i].name);
- }
+ img_info_list[i].flag.needs_rebasing = 1;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because it's base has changed (due to being reinstalled?)\n", img_info_list[i].name);
+ /* Set cur_base to the old base to simplify subsequent tests. */
+ cur_base = img_info_list[i].base;
+ }
+ /* However, if the DLL got bigger and doesn't fit into its slot
+ anymore, rebase this DLL from scratch. */
+ if (i + 1 < img_info_rebase_start
+ && cur_base + slot_size + offset > img_info_list[i + 1].base)
+ {
+ img_info_list[i].base = 0;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because it won't fit in it's old slot without overlapping next DLL\n", img_info_list[i].name);
+ }
+ /* Does the previous DLL reach into the address space of this
+ DLL? This happens if the previous DLL is not rebaseable. */
+ else if (i > 0 && cur_base < img_info_list[i - 1].base
+ + img_info_list[i - 1].slot_size)
+ {
+ img_info_list[i].base = 0;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because previous DLL now overlaps\n", img_info_list[i].name);
+ }
+ /* Does the file match the base address requirements? If not,
+ rebase from scratch. */
+ else if ((down_flag && cur_base + slot_size + offset > image_base)
+ || (!down_flag && cur_base < image_base))
+ {
+ img_info_list[i].base = 0;
+ if (verbose)
+ fprintf (stderr, "rebasing %s because it's base address is outside the expected area\n", img_info_list[i].name);
}
- /* Unconditionally overwrite old with new size. */
- img_info_list[i].size = cur_size;
- img_info_list[i].slot_size = slot_size;
/* Make sure all DLLs with base address 0 have the needs_rebasing
flag set. */
if (img_info_list[i].base == 0)
img_info_list[i].flag.needs_rebasing = 1;
+ /* Only check for writability if file needs rebasing to keep
+ Compact OS compression of unchanged files. Revert rebase
+ decision if not writeable. */
+ if (img_info_list[i].flag.needs_rebasing && set_cannot_rebase (&img_info_list[i]))
+ {
+ img_info_list[i].flag.needs_rebasing = 0;
+ img_info_list[i].base = cur_base_orig;
+ if (verbose)
+ fprintf (stderr, "rebasing %s deferred because file is not writable\n", img_info_list[i].name);
+ }
+ /* Unconditionally overwrite old with new size. */
+ img_info_list[i].size = cur_size;
+ img_info_list[i].slot_size = slot_size;
}
/* The remainder of the function expects img_info_size to be > 0. */
if (img_info_size == 0)
@@ -1380,6 +1408,79 @@ string_to_ulonglong (const char *string)
return number;
}
+static int
+compactos_get_algorithm (const char *pathname)
+{
+ /* Requires Win10. */
+ if (!IsWindows10OrGreater ())
+ return -1;
+
+ int fd = open (pathname, O_RDONLY);
+ if (fd == -1)
+ return -1;
+
+ struct {
+ WOF_EXTERNAL_INFO Wof;
+ FILE_PROVIDER_EXTERNAL_INFO_V1 FileProvider;
+ } wfp = { {0, 0}, {0, 0, 0} };
+
+ int rc;
+ if (!DeviceIoControl ((HANDLE) _get_osfhandle (fd),
+ FSCTL_GET_EXTERNAL_BACKING, NULL, 0,
+ &wfp, sizeof(wfp), NULL, NULL))
+ rc = -1;
+ else
+ rc = (wfp.Wof.Provider == WOF_PROVIDER_FILE ?
+ wfp.FileProvider.Algorithm : -1);
+
+ close (fd);
+ return rc;
+}
+
+static int
+compactos_compress_file (const char *pathname, int algorithm)
+{
+ /* Do not apply unknown algorithms. */
+ if (!(FILE_PROVIDER_COMPRESSION_XPRESS4K <= algorithm &&
+ algorithm <= FILE_PROVIDER_COMPRESSION_XPRESS16K))
+ return 0;
+
+ int fd = open (pathname, O_RDONLY);
+ if (fd == -1)
+ return -1;
+ HANDLE h = (HANDLE) _get_osfhandle (fd);
+
+ /* Older versions of Win10 set mtime to current time. */
+ FILETIME ft;
+ BOOL ft_valid = GetFileTime (h, NULL, NULL, &ft);
+
+ struct {
+ WOF_EXTERNAL_INFO Wof;
+ FILE_PROVIDER_EXTERNAL_INFO_V1 FileProvider;
+ } wfp;
+ wfp.Wof.Version = WOF_CURRENT_VERSION;
+ wfp.Wof.Provider = WOF_PROVIDER_FILE;
+ wfp.FileProvider.Version = FILE_PROVIDER_CURRENT_VERSION;
+ wfp.FileProvider.Algorithm = algorithm;
+ wfp.FileProvider.Flags = 0;
+
+ int rc;
+ if (!DeviceIoControl (h, FSCTL_SET_EXTERNAL_BACKING, &wfp, sizeof(wfp),
+ NULL, 0, NULL, NULL))
+ rc = (GetLastError() == ERROR_COMPRESSION_NOT_BENEFICIAL ? 0 : -1);
+ else
+ rc = 1;
+
+ if (ft_valid)
+ SetFileTime (h, NULL, NULL, &ft);
+
+ close (fd);
+ if (verbose)
+ printf ("%s: Compact OS algorithm %d %s\n", pathname, algorithm,
+ (rc < 0 ? "FAILED" : rc == 0 ? "not applied " : "applied"));
+ return rc;
+}
+
void
usage ()
{
--
2.37.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH rebase] Add support for Compact OS compression for Cygwin
2022-07-14 12:12 ` Christian Franke
@ 2022-07-14 14:28 ` Corinna Vinschen
2022-07-14 14:41 ` Christian Franke
0 siblings, 1 reply; 6+ messages in thread
From: Corinna Vinschen @ 2022-07-14 14:28 UTC (permalink / raw)
To: cygwin-patches
On Jul 14 14:12, Christian Franke wrote:
> Corinna Vinschen wrote:
> > On Jul 14 12:02, Christian Franke wrote:
> > > [Sorry if this is the wrong list]
> > Yes, in theorie, but no worries. However...
>
> What is the correct list in theory ?-)
https://sourceware.org/cygwin-apps/ as the home of the "Cygwin-Apps"
is a good hint in itself ;)
> > Given compactos stuff is a OS thingy and not actually a Cygwin feature,
> > why do we need an ifdef CYGWIN?
>
> Mainly because I didn't test on MSYS and other (which ever these are)
> environments. This also requires a recent release of MinGW-w64 headers
> (>=10.0.0) which includes (my) Compact OS patch.
I don't think there's any "other".
> > > +#endif
> > This ifdef still makes sense, of course ...
>
> Could possibly also be enhanced to __MSYS__ and msys1.dll.
Not sure this makes sense. Does their installer support CompactOS?
>
>
> > ... and on first glance, the
> > remainder of the patch LGTM.
>
> Thanks. Attached is an alternative patch with most ifdefs removed.
LGTM. I'm not going to push it, yet, because... do you still want to
add the aforementioned MSYS support? If not, I'll just go ahead.
Thanks,
Corinna
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH rebase] Add support for Compact OS compression for Cygwin
2022-07-14 14:28 ` Corinna Vinschen
@ 2022-07-14 14:41 ` Christian Franke
2022-07-14 14:54 ` Corinna Vinschen
0 siblings, 1 reply; 6+ messages in thread
From: Christian Franke @ 2022-07-14 14:41 UTC (permalink / raw)
To: cygwin-patches
Corinna Vinschen wrote:
> On Jul 14 14:12, Christian Franke wrote:
>
>>>> +#endif
>>> This ifdef still makes sense, of course ...
>> Could possibly also be enhanced to __MSYS__ and msys1.dll.
> Not sure this makes sense. Does their installer support CompactOS?
No, AFIAK. Then only (nonexistent?) users who run 'compact /c /exe:lzx
...' manually on their installation would benefit.
>>
>>> ... and on first glance, the
>>> remainder of the patch LGTM.
>> Thanks. Attached is an alternative patch with most ifdefs removed.
> LGTM. I'm not going to push it, yet, because... do you still want to
> add the aforementioned MSYS support? If not, I'll just go ahead.
Please go ahead. I don't want to add platform specific code not actually
tested on that platform.
Thanks,
Christian
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH rebase] Add support for Compact OS compression for Cygwin
2022-07-14 14:41 ` Christian Franke
@ 2022-07-14 14:54 ` Corinna Vinschen
0 siblings, 0 replies; 6+ messages in thread
From: Corinna Vinschen @ 2022-07-14 14:54 UTC (permalink / raw)
To: cygwin-patches
On Jul 14 16:41, Christian Franke wrote:
> Corinna Vinschen wrote:
> > On Jul 14 14:12, Christian Franke wrote:
> >
> > > > > +#endif
> > > > This ifdef still makes sense, of course ...
> > > Could possibly also be enhanced to __MSYS__ and msys1.dll.
> > Not sure this makes sense. Does their installer support CompactOS?
>
> No, AFIAK. Then only (nonexistent?) users who run 'compact /c /exe:lzx ...'
> manually on their installation would benefit.
>
>
> > >
> > > > ... and on first glance, the
> > > > remainder of the patch LGTM.
> > > Thanks. Attached is an alternative patch with most ifdefs removed.
> > LGTM. I'm not going to push it, yet, because... do you still want to
> > add the aforementioned MSYS support? If not, I'll just go ahead.
>
> Please go ahead. I don't want to add platform specific code not actually
> tested on that platform.
Pushed.
Thanks,
Corinna
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-07-14 14:54 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-14 10:02 [PATCH rebase] Add support for Compact OS compression for Cygwin Christian Franke
2022-07-14 10:24 ` Corinna Vinschen
2022-07-14 12:12 ` Christian Franke
2022-07-14 14:28 ` Corinna Vinschen
2022-07-14 14:41 ` Christian Franke
2022-07-14 14:54 ` Corinna Vinschen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).