From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 8129 invoked by alias); 13 Jul 2004 09:49:15 -0000 Mailing-List: contact ecos-discuss-help@ecos.sourceware.org; run by ezmlm Precedence: bulk List-Subscribe: List-Archive: List-Post: List-Help: , Sender: ecos-discuss-owner@ecos.sourceware.org Received: (qmail 8121 invoked from network); 13 Jul 2004 09:49:13 -0000 Received: from unknown (HELO mail.broadpark.no) (217.13.4.2) by sourceware.org with SMTP; 13 Jul 2004 09:49:13 -0000 Received: from famine (217-13-20-38.dd.nextgentel.com [217.13.20.38]) by mail.broadpark.no (Postfix) with ESMTP id A78E01B6B; Tue, 13 Jul 2004 11:49:42 +0200 (MEST) From: =?ISO-8859-1?Q?=D8yvind?= Harboe To: David Woodhouse Cc: ecos-discuss@sources.redhat.com In-Reply-To: <1089711000.2899.96.camel@hades.cambridge.redhat.com> References: <1089643331.3951.42.camel@famine> <1089711000.2899.96.camel@hades.cambridge.redhat.com> Content-Type: text/plain; charset=iso-8859-1 Message-Id: <1089712151.5995.21.camel@famine> Mime-Version: 1.0 Date: Tue, 13 Jul 2004 09:49:00 -0000 Content-Transfer-Encoding: 8bit Subject: Re: [ECOS] JFFS2 eats memory X-SW-Source: 2004-07/txt/msg00173.txt.bz2 On Tue, 2004-07-13 at 11:30, David Woodhouse wrote: > On Mon, 2004-07-12 at 16:42 +0200, Øyvind Harboe wrote: > > This issue has been discussed before, and although I have a workaround, > > I'd dearly like to have it put to bed since it is starting to cause > > problems elsewhere in my application: > > > > - My code opens a file for writing with O_TRUNC set, performs > > a single write call, closes the file. > > - After closing the file, JFFS2 has eaten memory. > > - With the attached modifcations to JFFS2, it "only" eats 24 bytes. > > That would be a single struct jffs2_raw_node_ref. That's 16 bytes and I > assume the other 8 is allocator overhead -- unless you're on a 64-bit > platform where it is actually 24 bytes. Yes. > It wouldn't be hard to cut the jffs2_raw_node_ref down to 12 bytes > instead of 16, btw. There are _many_ of these. See the discussion about > making the commented ref_totlen() macro in nodelist.h actually work in > 100% of cases, rather than only 95% as it would at the moment, and then > we could drop the totlen field. Changing the size of jffs2_raw_node_ref does not help much, since the problem is that my system runs out of memory since it continously overwrites existing files, thus filling up the flash with obsoleted nodes. > > Presumably when the raw nodes in the file fragement list are marked as > > obsolete, they are no longer required, but are not freed. > > The jffs2_raw_node_ref structures _are_ required. You need those in > order to be able to find all the parts of a given inode if/when it gets > opened again. > > But the file fragment list, which is held in memory for as long as the > eCos-specific "struct _inode" exists, is not necessarily required. I'm pretty sure the problem is in the file fragment list... > You've made the eCos code prune its inode cache more aggressively. Be > careful that you don't pessimise the garbage-collection code. It will > often spend a fair bit of time opening an inode (with > jffs2_gc_fetch_inode()), doing some garbage collection, and then > releasing it again (with jffs2_gc_release_inode()). Once/if I get the file fragment memory loss fixed, I'll try to remove it and run some stress testing for steady state memory usage to see if I can take out those changes. > > Now that we have those methods, actually, you could mark only _those_ > inodes so that they remain in cache for a little while longer, while > instantly releasing inodes from your icache if they were opened by the > user. > > > Q: Is this fundamentally impossible or a "bad idea" to fix? > > Not at all. Everything under fs/ecos/ is yours to do with as you see > fit. I'm happy to advise but I'd _really_ like to see someone rewrite > the eCos parts altogether; they still look _far_ too much like a > hacked-up Linux compatibility layer, and they no longer need to. Thats a lot more than I want to attempt with my current knowledge about JFFS2. :-) -- Øyvind Harboe http://www.zylin.com -- Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss