Why does ldd not show cyg*.dll in its output?

Why does ldd not show cyg*.dll in its output?

[Warren Young]

STC:

    $ ldd `which ls`

Actual output:

$ ldd `which ls`
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffd16fb0000)
        KERNEL32.DLL => /c/WINDOWS/system32/KERNEL32.DLL (0x7ffd16b80000)
        KERNELBASE.dll => /c/WINDOWS/system32/KERNELBASE.dll (0x7ffd13f50000)

Expected output:

According to Microsoft’s Dependency Walker tool, the output should also list cygwin1.dll and cygintl-8.dll, at minimum.  Since it seems happy to chase dependencies from kernel32.dll to the other two, which are not explicit dependencies of ls.exe, it should also list cygiconv-2.dll, via cygintl.

Bonus points if the output changes to a tree view, so the indirect dependencies are clear.
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Why does ldd not show cyg*.dll in its output?

[Andrey Repin]

Greetings, Warren Young!

> STC:

>     $ ldd `which ls`

> Actual output:

> $ ldd `which ls`
>         ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffd16fb0000)
>         KERNEL32.DLL => /c/WINDOWS/system32/KERNEL32.DLL (0x7ffd16b80000)
>         KERNELBASE.dll => /c/WINDOWS/system32/KERNELBASE.dll (0x7ffd13f50000)

> Expected output:

> According to Microsoft’s Dependency Walker tool, the output should also
> list cygwin1.dll and cygintl-8.dll, at minimum.  Since it seems happy to
> chase dependencies from kernel32.dll to the other two, which are not
> explicit dependencies of ls.exe, it should also list cygiconv-2.dll, via cygintl.

> Bonus points if the output changes to a tree view, so the indirect dependencies are clear.

$ cygcheck $(which ls)
C:\Programs\Cygwin_64\bin\ls.exe
  C:\Programs\Cygwin_64\bin\cygwin1.dll
    C:\Windows\system32\KERNEL32.dll
      C:\Windows\system32\API-MS-Win-Core-RtlSupport-L1-1-0.dll
      C:\Windows\system32\ntdll.dll
      C:\Windows\system32\KERNELBASE.dll
      C:\Windows\system32\API-MS-Win-Core-ProcessThreads-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Heap-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Memory-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Handle-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Synch-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-File-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-IO-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-ThreadPool-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-LibraryLoader-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-NamedPipe-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Misc-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-SysInfo-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Localization-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-ProcessEnvironment-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-String-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Debug-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-ErrorHandling-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Fibers-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Util-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Core-Profile-L1-1-0.dll
      C:\Windows\system32\API-MS-Win-Security-Base-L1-1-0.dll
  C:\Programs\Cygwin_64\bin\cygintl-8.dll
    C:\Programs\Cygwin_64\bin\cygiconv-2.dll

m?


-- 
With best regards,
Andrey Repin
Monday, May 16, 2016 19:47:22

Sorry for my terrible english...

Re: Why does ldd not show cyg*.dll in its output?

[Warren Young]

On May 16, 2016, at 10:48 AM, Andrey Repin <anrdaemon@yandex.ru> wrote:
> 
> Greetings, Warren Young!
> 
>> STC:
> 
>>    $ ldd `which ls`
> 
> $ cygcheck $(which ls)

Good to know.  But ldd should do it, too. :)

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Why does ldd not show cyg*.dll in its output?

[Yaakov Selkowitz]

On 2016-05-16 10:42, Warren Young wrote:
> $ ldd `which ls`
>         ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffd16fb0000)
>         KERNEL32.DLL => /c/WINDOWS/system32/KERNEL32.DLL (0x7ffd16b80000)
>         KERNELBASE.dll => /c/WINDOWS/system32/KERNELBASE.dll (0x7ffd13f50000)

WFM:

$ /bin/ldd /bin/ls
         ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77c90000)
         kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll 
(0x77a70000)
         KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll 
(0x7fefdb10000)
         cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000)
         cygintl-8.dll => /usr/bin/cygintl-8.dll (0x3d4e00000)
         cygiconv-2.dll => /usr/bin/cygiconv-2.dll (0x3d8b50000)

> Expected output:
>
> According to Microsoft’s Dependency Walker tool, the output should also list
> cygwin1.dll and cygintl-8.dll, at minimum.  Since it seems happy to chase
> dependencies from kernel32.dll to the other two, which are not explicit
> dependencies of ls.exe, it should also list cygiconv-2.dll, via cygintl.

See above.

> Bonus points if the output changes to a tree view, so the indirect dependencies are clear.

ldd is meant to be compatible with the Linux tool.  As Andrey mentioned, 
use cygcheck instead if you want that layout.

-- 
Yaakov

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Why does ldd not show cyg*.dll in its output?

[Hans-Bernhard Bröker]

Am 16.05.2016 um 19:50 schrieb Yaakov Selkowitz:
> On 2016-05-16 10:42, Warren Young wrote:
>> $ ldd `which ls`
>>         ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffd16fb0000)
>>         KERNEL32.DLL => /c/WINDOWS/system32/KERNEL32.DLL (0x7ffd16b80000)
>>         KERNELBASE.dll => /c/WINDOWS/system32/KERNELBASE.dll
>> (0x7ffd13f50000)
>
> WFM:
>
> $ /bin/ldd /bin/ls
>         ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77c90000)
>         kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll
> (0x77a70000)
>         KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll
> (0x7fefdb10000)
>         cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000)
>         cygintl-8.dll => /usr/bin/cygintl-8.dll (0x3d4e00000)
>         cygiconv-2.dll => /usr/bin/cygiconv-2.dll (0x3d8b50000)

Does not WFM:

$ ldd /usr/bin/ls.exe
         ntdll.dll => /cygdrive/c/windows/SYSTEM32/ntdll.dll 
(0x7ffe93c00000)
         KERNEL32.DLL => /cygdrive/c/windows/system32/KERNEL32.DLL 
(0x7ffe912d0000)
         KERNELBASE.dll => /cygdrive/c/windows/system32/KERNELBASE.dll 
(0x7ffe90310000)

(no different results from /bin/ls or /bin/ls.exe, either).

This is with ldd.exe from cygwin-2.5.1-1, on Win10 64bit, installed into 
c:\cygwin64



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Why does ldd not show cyg*.dll in its output?

[Warren Young]

On May 19, 2016, at 5:21 PM, Hans-Bernhard Bröker <HBBroeker@t-online.de> wrote:
> 
> Does not WFM:

[snip]

> This is with ldd.exe from cygwin-2.5.1-1, on Win10 64bit, installed into c:\cygwin64

That’s identical to my system.

I was going to attach the output of “strace ldd `which ls`” but that leaks too much detail about my system.  Any idea what I should be looking for in it?  I get about 400 lines of mud.
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Why does ldd not show cyg*.dll in its output?

[David Macek]

[-- Attachment #1: Type: text/plain, Size: 1467 bytes --]

On 20. 5. 2016 3:20, Warren Young wrote:
> On May 19, 2016, at 5:21 PM, Hans-Bernhard Bröker <HBBroeker@t-online.de> wrote:
>>
>> Does not WFM:
> 
> [snip]
> 
>> This is with ldd.exe from cygwin-2.5.1-1, on Win10 64bit, installed into c:\cygwin64

Just bumped into this as well.

Win10 64bit (1607 build 14367) with MSYS2 64bit:

$ ldd /usr/bin/ls
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7fff19e60000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7fff198a0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7fff17240000)

Win8.x 64bit with MSYS2 64bit:

$ ldd /usr/bin/ls
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffda3810000)
        KERNEL32.DLL => /c/Windows/system32/KERNEL32.DLL (0x7ffda3310000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7ffda0d80000)
        msys-intl-8.dll => /usr/bin/msys-intl-8.dll (0x430b30000)
        msys-2.0.dll => /usr/bin/msys-2.0.dll (0x180040000)
        msys-iconv-2.dll => /usr/bin/msys-iconv-2.dll (0x3ff760000)

Both installations are up to date and should have identical binaries.

Looking into procmon, the only relevant difference I see is `sysmain.sdb` being checked right after `ls.exe` process creation, but that could very well be a red herring. Unfortunately, ACT seems to be broken on Win10.

I can try watching them side by side in debuggers tomorrow, maybe I'll find something.

-- 
David Macek


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3834 bytes --]

Re: Why does ldd not show cyg*.dll in its output?

[Corinna Vinschen]

[-- Attachment #1: Type: text/plain, Size: 4326 bytes --]

Hi David,

On Jun 30 23:27, David Macek wrote:
> On 29. 6. 2016 23:45, David Macek wrote:
> > I can try watching them side by side in debuggers tomorrow, maybe I'll find something.
> 
> Yep, found something. TL;DR the issue is that Windows spins a thread
> in the process before our DLLs are loaded. Detailed analysis below.
> [...]
>  8. EXIT_PROCESS_DEBUG_EVENT
> 
> Event #6 is the culprit here. NT apparently starts a thread in our
> process which confuses ldd into believing that the process has
> finished loading DLL from its import table and kills it, as explained
> above. The thread appears non-deterministically, too, sometimes I see
> event #5 (and the associated DLL in ldd's output), sometimes not. The
> thread's entry point is always the same and ProcExp reports the same
> location (I checked in case of some symbol mash-up, as it often
> happens when debugging across the Cygwin-Windows boundary). When I
> broke ldd at `case CREATE_THREAD_DEBUG_EVENT`, the new thread's stack
> trace (from ProcExp) looked like this:
> [...]
> After some experimentation, I came up with a patch that tries to
> determine whether to ignore a thread based on the thread's entry point
> lying inside or outside of the memory occupied by ntdll.dll:

Nice detective work!  As for the below patch, I think the condition
should be turned around to test if the thread entry point is inside the
Cygwin DLL.  This event:

  10. CREATE_THREAD_DEBUG_EVENT ev.u.CreateThread = {hThread = 0x88, lpThreadLocalBase = 0x7ff5ffffb000, lpStartAddress = 0x180044cf0 <cygthread::stub(void*)>} 

denotes that we are ready.  So instead of contining if the thread entry
point is inside ntdll, continue as long as the entry point is not in the
Cygwin DLL.  That would also have the advantage to be independent of
future changes in Windows as well as injection from virus checkers etc.

What do you think?

> --- ldd.cc.orig 2016-04-28 21:54:57.556500700 +0200
> +++ ldd.cc      2016-06-30 23:24:20.394384800 +0200
> @@ -327,6 +327,10 @@
>      {
>        bool exitnow = false;
>        DWORD cont = DBG_CONTINUE;
> +      MODULEINFO mi;
> +      HMODULE ntdll = LoadLibrary("ntdll.dll");
> +      HMODULE ntdllend = NULL;
> +      void *entryPoint;
>        if (!WaitForDebugEvent (&ev, INFINITE))
>         break;
>        switch (ev.dwDebugEventCode)
> @@ -357,6 +361,31 @@
>             }
>           break;
>         case CREATE_THREAD_DEBUG_EVENT:
> +          if (ntdll != NULL)
> +            {
> +              if (ntdllend == NULL)
> +                {
> +                  /* Using our ntdll.dll HMODULE with a foreign process
> +                     should be fine because ntdll.dll is mapped to the same
> +                     address in every concurrently executing process and
> +                     HMODULEs are just pointers to the image in the process
> +                     memory. Using GetModuleHandle(NULL) didn't work for the
> +                     author of this code (-> ERROR_INVALID_HANDLE). */
> +                  if (GetModuleInformation(hProcess, ntdll, &mi, sizeof(mi)))
> +                    ntdllend = ntdll + (ptrdiff_t)mi.SizeOfImage;
> +                  else
> +                    ntdll = NULL;
> +                }
> +              if (ntdllend != NULL)
> +                {
> +                  entryPoint = (void*)ev.u.CreateThread.lpStartAddress;
> +                  /* If the thread's entry point is in ntdll.dll, let's
> +                    assume that this isn't the program's main thread and
> +                    ignore it. */
> +                  if (ntdll <= entryPoint && entryPoint < ntdllend)
> +                    break;
> +                }

This check can be simplified.  Rather than explicitly checking for start
and end address of ntdll.dll (or cygwin1.dll), one could just find the
file for a given address:

  wchar_t fnamebuf[MAX_PATH];
  if (GetMappedFileNameW (hProcess,
			  (PVOID) ev.u.CreateThread.lpStartAddress,
			  fnamebuf, sizeof fnamebuf)
      && wcsstr (fname, L"cygwin1.dll"))
    printf ("bingo\n");


Thanks,
Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Why does ldd not show cyg*.dll in its output?

[Marco Atzeri]

On 16/05/2016 17:42, Warren Young wrote:
> STC:
>
>     $ ldd `which ls`
>
> Actual output:
>
> $ ldd `which ls`
>         ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffd16fb0000)
>         KERNEL32.DLL => /c/WINDOWS/system32/KERNEL32.DLL (0x7ffd16b80000)
>         KERNELBASE.dll => /c/WINDOWS/system32/KERNELBASE.dll (0x7ffd13f50000)
>
> Expected output:
>
> According to Microsoft’s Dependency Walker tool, the output should also list cygwin1.dll and cygintl-8.dll, at minimum.  Since it seems happy to chase dependencies from kernel32.dll to the other two, which are not explicit dependencies of ls.exe, it should also list cygiconv-2.dll, via cygintl.
>
> Bonus points if the output changes to a tree view, so the indirect dependencies are clear.

it works as expected for me:

$ ldd /usr/bin/ls.exe
  ntdll.dll => /cygdrive/c/Windows/SYSTEM32/ntdll.dll (0x77410000)
  kernel32.dll => /cygdrive/c/Windows/system32/kernel32.dll (0x771f0000)
  KERNELBASE.dll => /cygdrive/c/Windows/system32/KERNELBASE.dll 
(0x7fefd460000)
  SYSFER.DLL => /cygdrive/c/Windows/System32/SYSFER.DLL (0x74e90000)
  cygwin1.dll => /usr/bin/cygwin1.dll (0x180040000)
  cygintl-8.dll => /usr/bin/cygintl-8.dll (0x3e1840000)
  cygiconv-2.dll => /usr/bin/cygiconv-2.dll (0x3e6e90000)


$ objdump -x /usr/bin/ls.exe |grep "DLL Name:"
         DLL Name: cygwin1.dll
         DLL Name: cygintl-8.dll
         DLL Name: KERNEL32.dll



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple