* GLIBC Localedata out of memory @ 2021-01-30 23:50 Stafford Horne 2021-01-31 0:55 ` Stafford Horne 2021-02-02 13:40 ` Adhemerval Zanella 0 siblings, 2 replies; 9+ messages in thread From: Stafford Horne @ 2021-01-30 23:50 UTC (permalink / raw) To: GLIBC patches Hello, I am working on testing the new OpenRISC port and getting all of the tests passing. I am having an issue with generating localedata on my board running out of memory. The process is getting killed by the linux oom-killer. The board has 256mb ram and it seems to not be enough. Lot's of ram is being used by my rootfs. Previously I was running tests on a simulator where I could adjust the ram to anything I like. But now I am running tests on my ARTY fpga dev board. Is it OK to skip locale generation in tests results? Or any suggestions? -Stafford ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-01-30 23:50 GLIBC Localedata out of memory Stafford Horne @ 2021-01-31 0:55 ` Stafford Horne 2021-02-02 13:40 ` Adhemerval Zanella 1 sibling, 0 replies; 9+ messages in thread From: Stafford Horne @ 2021-01-31 0:55 UTC (permalink / raw) To: GLIBC patches On Sun, Jan 31, 2021 at 08:50:20AM +0900, Stafford Horne wrote: > Hello, > > I am working on testing the new OpenRISC port and getting all of the tests > passing. I am having an issue with generating localedata on my board running > out of memory. The process is getting killed by the linux oom-killer. The > board has 256mb ram and it seems to not be enough. Lot's of ram is being used > by my rootfs. > > Previously I was running tests on a simulator where I could adjust the ram to > anything I like. But now I am running tests on my ARTY fpga dev board. > > Is it OK to skip locale generation in tests results? Or any suggestions? I am thinking swap of NFS. I'll give it a try. Also, I can move more of my rootfs (python, etc) over NFS too to free up memory. Just some initial idea's, I would like to get some tips of how other people run the glibc test suite on embedded systems. Do most developers just use simulators? -Staford ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-01-30 23:50 GLIBC Localedata out of memory Stafford Horne 2021-01-31 0:55 ` Stafford Horne @ 2021-02-02 13:40 ` Adhemerval Zanella 2021-02-02 21:40 ` Stafford Horne 1 sibling, 1 reply; 9+ messages in thread From: Adhemerval Zanella @ 2021-02-02 13:40 UTC (permalink / raw) To: Stafford Horne, GLIBC patches On 30/01/2021 20:50, Stafford Horne via Libc-alpha wrote: > Hello, > > I am working on testing the new OpenRISC port and getting all of the tests > passing. I am having an issue with generating localedata on my board running > out of memory. The process is getting killed by the linux oom-killer. The > board has 256mb ram and it seems to not be enough. Lot's of ram is being used > by my rootfs. I usually ran a full make check on a sh4 board that has 512MB of ram and it is shared with debian buildd process and I haven't see any issue. The machine does have 8GBi of swap enabled, so it might be mitigated by this. I checked the memory requirements of localedata generation (using valgrind massig) for tests and it seems that only one (cmw_TW.UTF8) consumes about ~190MB, mostly uses about ~150MB (as below). So I think using a small swap should help you here. az_AZ.UTF-8 : 143.4 MB bg_BG.UTF-8 : 145.29 MB am_ET.UTF-8 : 145.62 MB ber_DZ.UTF-8 : 145.28 MB ber_MA.UTF-8 : 145.26 MB be_BY.UTF-8 : 145.27 MB br_FR.UTF-8 : 145.26 MB bs_BA.UTF-8 : 145.27 MB ckb_IQ.UTF-8 : 144.6 MB cmn_TW.UTF-8 : 190.72 MB crh_UA.UTF-8 : 143.39 MB cs_CZ.UTF-8 : 145.24 MB csb_PL.UTF-8 : 145.28 MB cv_RU.UTF-8 : 145.26 MB cy_GB.UTF-8 : 145.3 MB da_DK.ISO-8859-1 : 68.19 MB de_DE.ISO-8859-1 : 68.36 MB de_DE.UTF-8 : 145.26 MB dsb_DE.UTF-8 : 144.62 MB dz_BT.UTF-8 : 145.98 MB en_GB.UTF-8 : 145.26 MB en_US.ANSI_X3.4-1968: 68.3 MB en_US.ISO-8859-1 : 68.36 MB en_US.UTF-8 : 145.26 MB eo.UTF-8 : 144.61 MB es_ES.UTF-8 : 145.26 MB et_EE.UTF-8 : 145.27 MB fa_IR.UTF-8 : 145.37 MB fi_FI.UTF-8 : 145.27 MB fil_PH.UTF-8 : 145.27 MB fr_CA.UTF-8 : 145.26 MB fr_FR.ISO-8859-1 : 68.36 MB fr_FR.UTF-8 : 145.26 MB fur_IT.UTF-8 : 145.26 MB gez_ER.UTF-8@abegede: 145.69 MB ha_NG.UTF-8 : 145.28 MB hr_HR.ISO-8859-2 : 68.38 MB hr_HR.UTF-8 : 145.27 MB hsb_DE.UTF-8 : 144.62 MB hu_HU.UTF-8 : 145.38 MB ig_NG.UTF-8 : 145.31 MB ik_CA.UTF-8 : 145.29 MB is_IS.UTF-8 : 145.27 MB ja_JP.EUC-JP : 34.47 MB ja_JP.SJIS : 31.63 MB ja_JP.UTF-8 : 96.93 MB kk_KZ.UTF-8 : 145.26 MB ku_TR.UTF-8 : 143.4 MB ky_KG.UTF-8 : 145.26 MB ln_CD.UTF-8 : 145.25 MB lt_LT.UTF-8 : 145.28 MB lv_LV.UTF-8 : 145.26 MB mi_NZ.UTF-8 : 145.26 MB ml_IN.UTF-8 : 145.31 MB mn_MN.UTF-8 : 145.29 MB mr_IN.UTF-8 : 145.3 MB mt_MT.UTF-8 : 145.26 MB nan_TW.UTF-8@latin : 145.26 MB nb_NO.ISO-8859-1 : 68.19 MB nb_NO.UTF-8 : 145.28 MB nl_NL.UTF-8 : 145.26 MB nn_NO.ISO-8859-1 : 68.39 MB om_KE.UTF-8 : 145.28 MB or_IN.UTF-8 : 145.33 MB os_RU.UTF-8 : 145.26 MB pl_PL.UTF-8 : 145.28 MB ps_AF.UTF-8 : 144.66 MB ro_RO.UTF-8 : 145.26 MB ru_RU.UTF-8 : 145.26 MB sah_RU.UTF-8 : 145.28 MB sc_IT.UTF-8 : 145.26 MB se_NO.UTF-8 : 145.28 MB si_LK.UTF-8 : 145.26 MB sq_AL.UTF-8 : 145.3 MB sr_RS.UTF-8 : 145.31 MB sv_SE.ISO-8859-1 : 68.2 MB sv_SE.UTF-8 : 145.27 MB szl_PL.UTF-8 : 145.28 MB tg_TJ.UTF-8 : 145.27 MB tk_TM.UTF-8 : 145.31 MB tr_TR.ISO-8859-9 : 66.72 MB tr_TR.UTF-8 : 143.4 MB tt_RU.UTF-8 : 145.27 MB tt_RU.UTF-8@iqtelif : 143.4 MB ug_CN.UTF-8 : 144.6 MB uk_UA.UTF-8 : 144.67 MB uz_UZ.UTF-8 : 145.31 MB vi_VN.UTF-8 : 145.26 MB yi_US.UTF-8 : 145.29 MB yo_NG.UTF-8 : 145.28 MB zh_CN.UTF-8 : 154.75 MB zh_TW.EUC-TW : 94.77 MB > > Previously I was running tests on a simulator where I could adjust the ram to > anything I like. But now I am running tests on my ARTY fpga dev board. > > Is it OK to skip locale generation in tests results? Or any suggestions? You will need the locales to get a full result, otherwise some tests will fail. I think it should be possible to build a native localedef and generate the locales in the host for a cross-compilation. You will need to handle the possible endianess mismatch though and there is the potential issue of a conversion library not support by the host that test might use. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-02-02 13:40 ` Adhemerval Zanella @ 2021-02-02 21:40 ` Stafford Horne 2021-05-29 10:08 ` Stafford Horne 0 siblings, 1 reply; 9+ messages in thread From: Stafford Horne @ 2021-02-02 21:40 UTC (permalink / raw) To: Adhemerval Zanella; +Cc: GLIBC patches On Tue, Feb 02, 2021 at 10:40:02AM -0300, Adhemerval Zanella wrote: > > > On 30/01/2021 20:50, Stafford Horne via Libc-alpha wrote: > > Hello, > > > > I am working on testing the new OpenRISC port and getting all of the tests > > passing. I am having an issue with generating localedata on my board running > > out of memory. The process is getting killed by the linux oom-killer. The > > board has 256mb ram and it seems to not be enough. Lot's of ram is being used > > by my rootfs. > > I usually ran a full make check on a sh4 board that has 512MB of ram and it is > shared with debian buildd process and I haven't see any issue. The machine does > have 8GBi of swap enabled, so it might be mitigated by this. > > I checked the memory requirements of localedata generation (using valgrind massig) > for tests and it seems that only one (cmw_TW.UTF8) consumes about ~190MB, mostly > uses about ~150MB (as below). So I think using a small swap should help you here. > > az_AZ.UTF-8 : 143.4 MB > bg_BG.UTF-8 : 145.29 MB > am_ET.UTF-8 : 145.62 MB > ber_DZ.UTF-8 : 145.28 MB > ber_MA.UTF-8 : 145.26 MB > be_BY.UTF-8 : 145.27 MB > br_FR.UTF-8 : 145.26 MB > bs_BA.UTF-8 : 145.27 MB > ckb_IQ.UTF-8 : 144.6 MB > cmn_TW.UTF-8 : 190.72 MB Actually, I was able to get the locale generation working yesterday by shrinking my rootfs as much as possible. Next I found that the sort-test is failing (localedata/xfrm-test with -nocache). The -nocache option makes the program allocate 4096 byte per each locale line to avoid strxfrm from using a cache path. Oh boy! > > > > Previously I was running tests on a simulator where I could adjust the ram to > > anything I like. But now I am running tests on my ARTY fpga dev board. > > > > Is it OK to skip locale generation in tests results? Or any suggestions? > > You will need the locales to get a full result, otherwise some tests will > fail. I think it should be possible to build a native localedef and > generate the locales in the host for a cross-compilation. You will need to > handle the possible endianess mismatch though and there is the potential > issue of a conversion library not support by the host that test might use. I think that I'm going to need to solve the ram issue by giving it some swap. I tried swap over NFS but that resulted in write errors (Write error on dio swapfile): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/page_io.c?h=v5.11-rc6#n351 I will try to use an SD card. -Stafford ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-02-02 21:40 ` Stafford Horne @ 2021-05-29 10:08 ` Stafford Horne 2021-06-01 17:45 ` Adhemerval Zanella 0 siblings, 1 reply; 9+ messages in thread From: Stafford Horne @ 2021-05-29 10:08 UTC (permalink / raw) To: Adhemerval Zanella; +Cc: GLIBC patches On Wed, Feb 03, 2021 at 06:40:34AM +0900, Stafford Horne wrote: > On Tue, Feb 02, 2021 at 10:40:02AM -0300, Adhemerval Zanella wrote: > > > > > > On 30/01/2021 20:50, Stafford Horne via Libc-alpha wrote: > > > Hello, > > > > > > I am working on testing the new OpenRISC port and getting all of the tests > > > passing. I am having an issue with generating localedata on my board running > > > out of memory. The process is getting killed by the linux oom-killer. The > > > board has 256mb ram and it seems to not be enough. Lot's of ram is being used > > > by my rootfs. > > > > I usually ran a full make check on a sh4 board that has 512MB of ram and it is > > shared with debian buildd process and I haven't see any issue. The machine does > > have 8GBi of swap enabled, so it might be mitigated by this. > > > > I checked the memory requirements of localedata generation (using valgrind massig) > > for tests and it seems that only one (cmw_TW.UTF8) consumes about ~190MB, mostly > > uses about ~150MB (as below). So I think using a small swap should help you here. > > > > az_AZ.UTF-8 : 143.4 MB > > bg_BG.UTF-8 : 145.29 MB > > am_ET.UTF-8 : 145.62 MB > > ber_DZ.UTF-8 : 145.28 MB > > ber_MA.UTF-8 : 145.26 MB > > be_BY.UTF-8 : 145.27 MB > > br_FR.UTF-8 : 145.26 MB > > bs_BA.UTF-8 : 145.27 MB > > ckb_IQ.UTF-8 : 144.6 MB > > cmn_TW.UTF-8 : 190.72 MB > > Actually, I was able to get the locale generation working yesterday by shrinking > my rootfs as much as possible. Next I found that the sort-test is failing > (localedata/xfrm-test with -nocache). The -nocache option makes the program > allocate 4096 byte per each locale line to avoid strxfrm from using a cache > path. Oh boy! > > > > > > > Previously I was running tests on a simulator where I could adjust the ram to > > > anything I like. But now I am running tests on my ARTY fpga dev board. > > > > > > Is it OK to skip locale generation in tests results? Or any suggestions? > > > > You will need the locales to get a full result, otherwise some tests will > > fail. I think it should be possible to build a native localedef and > > generate the locales in the host for a cross-compilation. You will need to > > handle the possible endianess mismatch though and there is the potential > > issue of a conversion library not support by the host that test might use. > > I think that I'm going to need to solve the ram issue by giving it some swap. > > I tried swap over NFS but that resulted in write errors (Write error on dio > swapfile): > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/page_io.c?h=v5.11-rc6#n351 > > I will try to use an SD card. So, this took a bit longer than I expected but I have swap working now over SD card and I don't have a memory issue any more generating locale data. Interesting chain of events: - Had to order SD card pmod for my board which took a few days - Next, I found my SoC had broken hardware (ok its FPGA, but it took about a month to fix.) - After fixing we found some issues with the kernel driver, but now those are fixed and everyting works fine. - My workstation died so I had to rebuild it, now I have a fancy 6 core processor. Anyway, another question. Do most glibc embedded developers build and run tests on embedded boards using a cross-compiler and 'scripts/cross-test-ssh.sh'? I have been running tests like this using a buildroot rootfs on my board. I now have 58 failures, but looking into them some seem related to handling of stdout and it seems like maybe its related to running over SSH. If this is what other are doing then I will continue with SSH and figure out these issues. But if other do another way, i.e. with a native compiler on the board then I will look into that. In that case I will probably have to start building debian for fedora as buildroot doesn't allow for creating a native compiler. -Stafford ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-05-29 10:08 ` Stafford Horne @ 2021-06-01 17:45 ` Adhemerval Zanella 2021-06-01 18:01 ` Joseph Myers 2021-06-03 10:37 ` Stafford Horne 0 siblings, 2 replies; 9+ messages in thread From: Adhemerval Zanella @ 2021-06-01 17:45 UTC (permalink / raw) To: Stafford Horne; +Cc: GLIBC patches On 29/05/2021 07:08, Stafford Horne wrote: > On Wed, Feb 03, 2021 at 06:40:34AM +0900, Stafford Horne wrote: >> On Tue, Feb 02, 2021 at 10:40:02AM -0300, Adhemerval Zanella wrote: >>> >>> >>> On 30/01/2021 20:50, Stafford Horne via Libc-alpha wrote: >>>> Hello, >>>> >>>> I am working on testing the new OpenRISC port and getting all of the tests >>>> passing. I am having an issue with generating localedata on my board running >>>> out of memory. The process is getting killed by the linux oom-killer. The >>>> board has 256mb ram and it seems to not be enough. Lot's of ram is being used >>>> by my rootfs. >>> >>> I usually ran a full make check on a sh4 board that has 512MB of ram and it is >>> shared with debian buildd process and I haven't see any issue. The machine does >>> have 8GBi of swap enabled, so it might be mitigated by this. >>> >>> I checked the memory requirements of localedata generation (using valgrind massig) >>> for tests and it seems that only one (cmw_TW.UTF8) consumes about ~190MB, mostly >>> uses about ~150MB (as below). So I think using a small swap should help you here. >>> >>> az_AZ.UTF-8 : 143.4 MB >>> bg_BG.UTF-8 : 145.29 MB >>> am_ET.UTF-8 : 145.62 MB >>> ber_DZ.UTF-8 : 145.28 MB >>> ber_MA.UTF-8 : 145.26 MB >>> be_BY.UTF-8 : 145.27 MB >>> br_FR.UTF-8 : 145.26 MB >>> bs_BA.UTF-8 : 145.27 MB >>> ckb_IQ.UTF-8 : 144.6 MB >>> cmn_TW.UTF-8 : 190.72 MB >> >> Actually, I was able to get the locale generation working yesterday by shrinking >> my rootfs as much as possible. Next I found that the sort-test is failing >> (localedata/xfrm-test with -nocache). The -nocache option makes the program >> allocate 4096 byte per each locale line to avoid strxfrm from using a cache >> path. Oh boy! >> >>>> >>>> Previously I was running tests on a simulator where I could adjust the ram to >>>> anything I like. But now I am running tests on my ARTY fpga dev board. >>>> >>>> Is it OK to skip locale generation in tests results? Or any suggestions? >>> >>> You will need the locales to get a full result, otherwise some tests will >>> fail. I think it should be possible to build a native localedef and >>> generate the locales in the host for a cross-compilation. You will need to >>> handle the possible endianess mismatch though and there is the potential >>> issue of a conversion library not support by the host that test might use. >> >> I think that I'm going to need to solve the ram issue by giving it some swap. >> >> I tried swap over NFS but that resulted in write errors (Write error on dio >> swapfile): >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/page_io.c?h=v5.11-rc6#n351 >> >> I will try to use an SD card. > > So, this took a bit longer than I expected but I have swap working now over SD > card and I don't have a memory issue any more generating locale data. > > Interesting chain of events: > - Had to order SD card pmod for my board which took a few days > - Next, I found my SoC had broken hardware (ok its FPGA, but it took about > a month to fix.) > - After fixing we found some issues with the kernel driver, but now those > are fixed and everyting works fine. > - My workstation died so I had to rebuild it, now I have a fancy 6 core > processor. > > Anyway, another question. Do most glibc embedded developers build and run tests > on embedded boards using a cross-compiler and 'scripts/cross-test-ssh.sh'? I usually run it natively, but I think both recent ports for arc and riscv32 were done using the scripts/cross-test-ssh.sh. > > I have been running tests like this using a buildroot rootfs on my board. > > I now have 58 failures, but looking into them some seem related to handling of > stdout and it seems like maybe its related to running over SSH. > > If this is what other are doing then I will continue with SSH and figure out > these issues. But if other do another way, i.e. with a native compiler on > the board then I will look into that. In that case I will probably have to > start building debian for fedora as buildroot doesn't allow for creating > a native compiler. Usually the issues reported for bring up and newer architectures are: - Tests timeout. A lot of tests are build and set to run on faster machine, so they might timeout on slower ones. I would recommend to run the testsuite with TIMEOUTFACTOR environment variable set to 10 or something higher (it might extend the make check if the test is stuck due some failure). - Missing installed library, either libgcc_s.so or libstdc++.so. The former is required for stack unwind required for some pthread routines while the latter for testing. I would recommend to copy both of them to the build directory prior ran the testsuite. - Issues with file permission or synchronization issues depending of the filesystem. If I recall correctly NFS might not implement all the required facilities depending of the version used. I am not sure how to mitigate it. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-06-01 17:45 ` Adhemerval Zanella @ 2021-06-01 18:01 ` Joseph Myers 2021-06-03 10:35 ` Stafford Horne 2021-06-03 10:37 ` Stafford Horne 1 sibling, 1 reply; 9+ messages in thread From: Joseph Myers @ 2021-06-01 18:01 UTC (permalink / raw) To: Adhemerval Zanella; +Cc: Stafford Horne, GLIBC patches On Tue, 1 Jun 2021, Adhemerval Zanella via Libc-alpha wrote: > - Issues with file permission or synchronization issues depending of > the filesystem. If I recall correctly NFS might not implement all > the required facilities depending of the version used. I am not > sure how to mitigate it. Mounting with acdirmax=0,acdirmin=0 may help with some synchronization issues. -- Joseph S. Myers joseph@codesourcery.com ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-06-01 18:01 ` Joseph Myers @ 2021-06-03 10:35 ` Stafford Horne 0 siblings, 0 replies; 9+ messages in thread From: Stafford Horne @ 2021-06-03 10:35 UTC (permalink / raw) To: Joseph Myers; +Cc: Adhemerval Zanella, GLIBC patches On Tue, Jun 01, 2021 at 06:01:53PM +0000, Joseph Myers wrote: > On Tue, 1 Jun 2021, Adhemerval Zanella via Libc-alpha wrote: > > > - Issues with file permission or synchronization issues depending of > > the filesystem. If I recall correctly NFS might not implement all > > the required facilities depending of the version used. I am not > > sure how to mitigate it. > > Mounting with acdirmax=0,acdirmin=0 may help with some synchronization > issues. Thanks, I think this is sorted out for me. I haven't seen NFS sync related issues. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: GLIBC Localedata out of memory 2021-06-01 17:45 ` Adhemerval Zanella 2021-06-01 18:01 ` Joseph Myers @ 2021-06-03 10:37 ` Stafford Horne 1 sibling, 0 replies; 9+ messages in thread From: Stafford Horne @ 2021-06-03 10:37 UTC (permalink / raw) To: Adhemerval Zanella; +Cc: GLIBC patches On Tue, Jun 01, 2021 at 02:45:47PM -0300, Adhemerval Zanella wrote: > > > On 29/05/2021 07:08, Stafford Horne wrote: > > On Wed, Feb 03, 2021 at 06:40:34AM +0900, Stafford Horne wrote: > >> On Tue, Feb 02, 2021 at 10:40:02AM -0300, Adhemerval Zanella wrote: > >>> > >>> > >>> On 30/01/2021 20:50, Stafford Horne via Libc-alpha wrote: > >>>> Hello, > >>>> > >>>> I am working on testing the new OpenRISC port and getting all of the tests > >>>> passing. I am having an issue with generating localedata on my board running > >>>> out of memory. The process is getting killed by the linux oom-killer. The > >>>> board has 256mb ram and it seems to not be enough. Lot's of ram is being used > >>>> by my rootfs. > >>> > >>> I usually ran a full make check on a sh4 board that has 512MB of ram and it is > >>> shared with debian buildd process and I haven't see any issue. The machine does > >>> have 8GBi of swap enabled, so it might be mitigated by this. > >>> > >>> I checked the memory requirements of localedata generation (using valgrind massig) > >>> for tests and it seems that only one (cmw_TW.UTF8) consumes about ~190MB, mostly > >>> uses about ~150MB (as below). So I think using a small swap should help you here. > >>> > >>> az_AZ.UTF-8 : 143.4 MB > >>> bg_BG.UTF-8 : 145.29 MB > >>> am_ET.UTF-8 : 145.62 MB > >>> ber_DZ.UTF-8 : 145.28 MB > >>> ber_MA.UTF-8 : 145.26 MB > >>> be_BY.UTF-8 : 145.27 MB > >>> br_FR.UTF-8 : 145.26 MB > >>> bs_BA.UTF-8 : 145.27 MB > >>> ckb_IQ.UTF-8 : 144.6 MB > >>> cmn_TW.UTF-8 : 190.72 MB > >> > >> Actually, I was able to get the locale generation working yesterday by shrinking > >> my rootfs as much as possible. Next I found that the sort-test is failing > >> (localedata/xfrm-test with -nocache). The -nocache option makes the program > >> allocate 4096 byte per each locale line to avoid strxfrm from using a cache > >> path. Oh boy! > >> > >>>> > >>>> Previously I was running tests on a simulator where I could adjust the ram to > >>>> anything I like. But now I am running tests on my ARTY fpga dev board. > >>>> > >>>> Is it OK to skip locale generation in tests results? Or any suggestions? > >>> > >>> You will need the locales to get a full result, otherwise some tests will > >>> fail. I think it should be possible to build a native localedef and > >>> generate the locales in the host for a cross-compilation. You will need to > >>> handle the possible endianess mismatch though and there is the potential > >>> issue of a conversion library not support by the host that test might use. > >> > >> I think that I'm going to need to solve the ram issue by giving it some swap. > >> > >> I tried swap over NFS but that resulted in write errors (Write error on dio > >> swapfile): > >> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/page_io.c?h=v5.11-rc6#n351 > >> > >> I will try to use an SD card. > > > > So, this took a bit longer than I expected but I have swap working now over SD > > card and I don't have a memory issue any more generating locale data. > > > > Interesting chain of events: > > - Had to order SD card pmod for my board which took a few days > > - Next, I found my SoC had broken hardware (ok its FPGA, but it took about > > a month to fix.) > > - After fixing we found some issues with the kernel driver, but now those > > are fixed and everyting works fine. > > - My workstation died so I had to rebuild it, now I have a fancy 6 core > > processor. > > > > Anyway, another question. Do most glibc embedded developers build and run tests > > on embedded boards using a cross-compiler and 'scripts/cross-test-ssh.sh'? > > I usually run it natively, but I think both recent ports for arc and riscv32 > were done using the scripts/cross-test-ssh.sh. Thanks, its good to know. I will update the wiki with these details: - https://sourceware.org/glibc/wiki/Testing/Testsuite#Testing_with_a_cross-compiler > > > > I have been running tests like this using a buildroot rootfs on my board. > > > > I now have 58 failures, but looking into them some seem related to handling of > > stdout and it seems like maybe its related to running over SSH. > > > > If this is what other are doing then I will continue with SSH and figure out > > these issues. But if other do another way, i.e. with a native compiler on > > the board then I will look into that. In that case I will probably have to > > start building debian for fedora as buildroot doesn't allow for creating > > a native compiler. > > Usually the issues reported for bring up and newer architectures are: > > - Tests timeout. A lot of tests are build and set to run on faster > machine, so they might timeout on slower ones. I would recommend > to run the testsuite with TIMEOUTFACTOR environment variable set > to 10 or something higher (it might extend the make check if the > test is stuck due some failure). I have sored this one, but it could be that some of the 58 failures are still just timeouts. > - Missing installed library, either libgcc_s.so or libstdc++.so. > The former is required for stack unwind required for some pthread > routines while the latter for testing. I would recommend to > copy both of them to the build directory prior ran the testsuite. Yup, I have sorted this one out. > - Issues with file permission or synchronization issues depending of > the filesystem. If I recall correctly NFS might not implement all > the required facilities depending of the version used. I am not > sure how to mitigate it. Thanks. Thanks for the notes. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-06-03 10:37 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-30 23:50 GLIBC Localedata out of memory Stafford Horne 2021-01-31 0:55 ` Stafford Horne 2021-02-02 13:40 ` Adhemerval Zanella 2021-02-02 21:40 ` Stafford Horne 2021-05-29 10:08 ` Stafford Horne 2021-06-01 17:45 ` Adhemerval Zanella 2021-06-01 18:01 ` Joseph Myers 2021-06-03 10:35 ` Stafford Horne 2021-06-03 10:37 ` Stafford Horne
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).