From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-74.mimecast.com (us-smtp-delivery-74.mimecast.com [63.128.21.74]) by sourceware.org (Postfix) with ESMTP id 0BECA3937430 for ; Wed, 18 Mar 2020 14:42:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 0BECA3937430 Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-54-_Q6lVY4aNASqxqdkGiPbPg-1; Wed, 18 Mar 2020 10:42:40 -0400 X-MC-Unique: _Q6lVY4aNASqxqdkGiPbPg-1 Received: by mail-qk1-f198.google.com with SMTP id e10so25887536qkg.8 for ; Wed, 18 Mar 2020 07:42:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=rPponp+YrgtidUijE9D3eE9AzgvlSSSiZ858azdLI3o=; b=B4MzHCHWz6tQE7UeQD19POwY/h946iNC9DvUtr3cwT+1lhy3XfB/J/3MsYsCAX3xOy E1tomuUYE4D1n+AT6RiF9YsoA4vqw3NYeLHugxiwfBawzQ73guUhAP4djzkeHSKKYi0s kA6nG54S5WpbcmBq+lPdpvVMTuCko1NQL7wCRTbjakeaFmXUyqy2796fjgINnY12DZKB UnwXc50iSS+rqJCuGlYx5/DYijr1hoW2Y3a8Gu2H0+m8ePyBOU99oWCtw2I5aQKOYPm8 +4AAdgE0EQSyr8rzaisGaIwhdWuvDFjVIhnqiP21RO9jzgxCUFhPtr4CSR/90Drs1u80 9a8Q== X-Gm-Message-State: ANhLgQ3+VxfgkvLEdiJtm83RxUDaDV6KSEzzGm2NyG/5zVv5KXZtXGVH /qUKTaVHi6jcdEVCNmutBeg16f4DEDt1kwHW1v8GzGTedZ21mCuYMfmiSDzqV6PIJOODrTksZJ1 FgqXn+RmlQXQ7hnyaNaa4 X-Received: by 2002:a05:6214:a63:: with SMTP id ef3mr4650858qvb.119.1584542558435; Wed, 18 Mar 2020 07:42:38 -0700 (PDT) X-Google-Smtp-Source: ADFU+vvd9hwzFQ6NNDxJcJBvZ3p8vLVehIjUN3NTpEk8x1QZqTw4Wq4bquKykV9KVgWUNWNgzvCWjQ== X-Received: by 2002:a05:6214:a63:: with SMTP id ef3mr4650823qvb.119.1584542558130; Wed, 18 Mar 2020 07:42:38 -0700 (PDT) Received: from [192.168.1.4] (198-84-170-103.cpe.teksavvy.com. [198.84.170.103]) by smtp.gmail.com with ESMTPSA id w18sm4093065qkw.130.2020.03.18.07.42.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 18 Mar 2020 07:42:37 -0700 (PDT) Subject: Re: pthread_cond performence Discussion To: Torvald Riegel , liqingqing , "libc-alpha@sourceware.org" Cc: Hushiyuan , Liusirui , wangshuo47@huawei.com References: <493791afc4761086757debcb7916fff852ea795f.camel@redhat.com> From: Carlos O'Donell Organization: Red Hat Message-ID: <1501a657-d8b8-7a7d-6f6a-ace0e9a3f435@redhat.com> Date: Wed, 18 Mar 2020 10:42:36 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: <493791afc4761086757debcb7916fff852ea795f.camel@redhat.com> Content-Language: en-US X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2020 14:42:43 -0000 On 3/18/20 8:53 AM, Torvald Riegel wrote: > On Wed, 2020-03-18 at 08:12 -0400, Carlos O'Donell wrote: >> On 3/16/20 3:30 AM, liqingqing wrote: >>> The new condvar implementation that provides stronger ordering >>> guarantees. For the waiters's ordering without expand the size of >>> the >>> struct of pthread_cond_t, It uses a little bits to maintain the >>> state >>> machine which has two different start group G1 and G2. This >>> algorithm >>> is very cleverly. But when I test MySQL performance and found that >>> this new condvar implementation will affect the performance when >>> there are many cores in one machine. the scenario is that in my arm >>> server, test 200 terminals to read and write the database in 4P >>> processor environment(totally 256 cores), and I found that It can >>> get >>> better performance when I use the old algorithm. >> >> Are you able to look at any hardware performance counters to see if >> there are increased cache line miss rates? >> >>> I think maybe there has too many cache false sharing when in my >>> environment. Does anyone has the same problem? And is there room >>> for >>> optimization about the new algorithm? >> >> I have not seen anyone report a performance problem on large >> machines. >> >> Unfortunately from an ABI perspective we cannot increase the size of >> the structure, nor change the required alignment. >> >> We may be able to play with the order of the layout of elements >> within the condvar. That's something you could experiment with and >> report back to the list with your findings. >> >> For example: >> - Make changes the layout by moving elements around to attempt to >> avoid cache-line sharing. >> - Recompile glibc. >> - Install into your system. >> - PTHREAD_COND_INITIALIZER should be all-zero bytes so you should >> not need to recompile applications. >> - Retest performance. > > The other thing I would recommend is to investigate whether you can > improve the synchronization in MySQL. Condition variables are just one > way to do synchronization, and the root synchronization problem to > solve is in the application. This could potentially give you a much > bigger performance improvement than any optimization of the condition > variable implementation could. Agreed. The ABI limitations on the current interface are the biggest problem. To fix the ABI issues would require structure allocation and freeing in the background, which has it's own issues and performance impact (requires fallback PSHARED handling). Therefore if you could parallelize differently then it would certainly help. I expect that this is harder than you think though since it requires refactoring the locking in mysql. While making the current condvar layout more cahceline friendly (even probablistically so) would improve all applications using it. -- Cheers, Carlos.