From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf1-x144.google.com (mail-lf1-x144.google.com [IPv6:2a00:1450:4864:20::144]) by sourceware.org (Postfix) with ESMTPS id B11873858037 for ; Wed, 6 Dec 2023 04:09:27 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org B11873858037 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org B11873858037 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::144 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701835769; cv=none; b=FhvZySz2aFio8SiLjkvEiKzkpewJZ1bbOpMUWYTh0PmKJB8jv1YsiRPj5ezHAvnaK0lqutt/IGVDJSJQizsuALlXfZMDS39Ln3Np731VLoCKK/IPkFC0Dyp5Bv6K62PcPz2kJ0NdVCHLbVxmMSJzQuWTcb+IOXHJ9gb6XHNzvYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1701835769; c=relaxed/simple; bh=jTfa7VzxSvW1E8jhkQFKWfXTJowrl26F/30UQcUOUoM=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=QucRAzMKxj4OUVT1Kw2LH1sQbQqO8W5m1WzVA4TcJqeb64e9boB69vE1Bc/zvsYI5npjFWWgCOXfScI0OwwJg2RlL1afwpQ22iNkDizPY9jkq0cEqEcFhcT39m/7EGMasp6UgdXjgsYHyd1Z7cshqG1Gq8zYGfN1IP0xj9SIJHo= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-lf1-x144.google.com with SMTP id 2adb3069b0e04-50bf82f4409so503213e87.0 for ; Tue, 05 Dec 2023 20:09:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701835766; x=1702440566; darn=cygwin.com; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=+Gb77FtCn8yM5TJlAgFjx0uV1KPbM7eyLHypJ0JXZxo=; b=Z4qpCXwD+waQxzXI7YOIq3FteBz8Gxatdg0VGCMch5gmIph3O3pfQwCNxhuK7JSYy2 uq9BXWUxuPI15Ient67ZD2VuMuJ7+2KV1TQIQ71WGwmBjMXyScn/qwfRnQwQgeuk9q0q JvW3n4jZpgHuJIIKzsPQxpQGrGB3CSqxzr+ypMbWIsHcyLGVF1xAQpbGs8A7ubLZjWSb +kVk+c1zdqkmrPo6BjMP5eYPbf38SHXc3XqX3Q459u+hHIRq4ur8Ngt2dAHyMhck7UK1 A/xzEP6PZ1OeF36QF6ctsxnX5LhqyvB6UwXuKO2571/KTRYz3jlgPKCGFgvSVlEaqbHV fJxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701835766; x=1702440566; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=+Gb77FtCn8yM5TJlAgFjx0uV1KPbM7eyLHypJ0JXZxo=; b=SHUzxDq1x/xp8EunuhL6pCo6W8iU+R2UPBD6D1AmARAjTWY8WfP7H17t+WI+HAlvu7 FdZYRN64SQe1/wrs8AH5sONtATKasxxfXT/13IdnyXZrNgyRsMh6otBfo5zPOFpZZBQH nH9Xm/xTb8sFn1z/P8mmbZ16wn8yF44qq5j07tgknyQMwEDwD9p7HHuFHQh8CGBDxwuG wX/CjrhKdOkcZaN7g2oDq4haDqb1v2vcg/WUcl4vHeqyMZosdL/GL9NEAljwocpSRJr7 4uM9cZpaga0SB47AhwG8z7U1dN76wKZtS3c/AdoShEEgIpIDj5i0hfVkGxr1sZewzFV7 R7Jw== X-Gm-Message-State: AOJu0YzME74uh3gXxZmqpbv95LFZPC9UdIso5WhmZC0D+hvMhzwhLdj9 51unpTVcceB7/T8kp9jJwRroeqSsJITJ0Y6qEuSrzc29bpIKPhO2+XY= X-Google-Smtp-Source: AGHT+IHElyujV1AAffCNgYRobgSp3volelXQ+Ee5bo6Npk1n5PQznAc9aVNaHxIvj181E+rQqPuP46VpVn/QuhURYL4= X-Received: by 2002:a05:6512:39cb:b0:50b:f88b:1680 with SMTP id k11-20020a05651239cb00b0050bf88b1680mr137624lfu.34.1701835765665; Tue, 05 Dec 2023 20:09:25 -0800 (PST) MIME-Version: 1.0 From: Dan Shelton Date: Wed, 6 Dec 2023 05:08:58 +0100 Message-ID: Subject: Catastrophic Cygwin find . -ls, grep performance on samba share compared to WSL&Linux To: cygwin@cygwin.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=1.3 required=5.0 tests=BAYES_05,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hello! I am unhappy to report a severe performance issue with find -ls, ls -R and grep -r, with Cygwin 3.4.9 and Cygwin 3.5.0 when samba shares are involved. Imagine a directory with 256 subdirs, and each has 256 files per subdir, all on a samba share, samba server is on Linux with tmpfs. mkdir dir1 for ((i=0;i<256;i++)) ; do mkdir "dir1/subdir$i" for ((j=0; j < 256;j++));do echo "j=$j" >"dir1/subdir$i/j$j.txt" done done Time comparisations then show a dramatic difference, Debian Linux accessing the samba share, WSL accessing the samba share, and Cygwin accessing the samba share: 1. time find . >/dev/null Cygwin 86 seconds WSL 23 seconds Debian 19 seconds 2. time find . -ls >/dev/null Cygwin 129 seconds WSL 38 seconds Debian 32 seconds 3. time grep -r -E NOMATCH 2>/dev/null Cygwin 390 seconds WSL 144 seconds Debian 141 seconds So where does the bad Cygwin performance come from? Virus checker, memory compression and other Windows services known to interfere with benchmarking are OFF. But the network trace shows a dramatic difference: While Debian and WSL open files only once, the Cygwin run spends lots of network traffic checking whether the txt files are txt.lnk, txt,bat.lnk and so on, all non existent files. Why does that happen? -- Dan Shelton - Cluster Specialist Win/Lin/Bsd