"User does not understand how *NIX systems utilize memory" is not the same thing as "bug". What you are seeing is the kernel using memory for file system cache. On 03/22/2001 12:58 -0800, Craig Cummings wrote: >> Hi there, >> >> I think this qualifies as a bug but let me know if this could be a >> configuration or hardware issue. >> >> I've been having problems with memory leaks when I run programs on >> large--up to 250MB text files. (I know this is huge, but that's the 3 >> billion base human genome for you.) At first I though it was a Perl >> problem but I later found that a completely unrelated C program also >> caused memory leaks. I recently upgraded to the 2.4 kernel, hoping to >> solve these problems (see below). However, the memory leaks are still >> happening and this time I know the problem is at a deeper level than my >> programs. Some standard UNIX programs are leaking a lot of memory. I >> would appreciate some advice and ultimately, a fix. Unfortunately, My >> programming skills are not sufficient for tinkering with the kernel >> source. Thank you, in advance for your help. Details follow. >> >> Regards, >> >> Craig Cummings >> >> >> Here are the specs for my system: >> Dell Precision XPSt700, Pentium III, 512 MB RAM >> I've recently upgraded from Red Hat 6.2 with the 2.2.14 kernel to >> Red Hat 7, then built the 2.4.2 kernel on my own. >> >> Here's what happens with grep: >> >> Output of free, freshly booted system: >> >> total used free shared buffers cached >> Mem: 513616 47516 466100 0 2476 27048 >> -/+ buffers/cache: 17992 495624 >> Swap: 128480 0 128480 >> >> Output of free after grep 'NT_005289' Data/hs_chr12.fa: >> >> total used free shared buffers cached >> Mem: 513616 183548 330068 0 2624 159616 >> -/+ buffers/cache: 21308 492308 >> Swap: 128480 0 128480 >> >> Output of grep 'NT_005289' Data/hs_chr2.fa: >> >> >gi|12728771|ref|NT_005289.2|Hs2_5446 Homo sapiens chromosome 2 working draft sequence segment >> >> Output of free after this grep: >> >> total used free shared buffers cached >> Mem: 513616 424272 89344 0 2860 394232 >> -/+ buffers/cache: 27180 486436 >> Swap: 128480 0 128480 >> >> Output of grep 'NT_005289' Data/hs_chr2.fa: >> >> >gi|12728771|ref|NT_005289.2|Hs2_5446 Homo sapiens chromosome 2 working draft sequence segment >> >> Output of free after this grep: >> >> total used free shared buffers cached >> Mem: 513616 424272 89344 0 2860 394232 >> -/+ buffers/cache: 27180 486436 >> Swap: 128480 0 128480 >> >> File sizes of the two files grep'ed: >> >> -rw-rw-r-- 1 cummings genomics 135744469 Mar 12 22:09 Data/hs_chr12.fa >> -rw-rw-r-- 1 cummings genomics 240244039 Mar 12 22:24 Data/hs_chr2.fa >> >> Note that these file sizes are equivalent to the amount of memory leaked >> when grep is called on that file. >> >> When I grep the same file a second time, very little additional memory is >> leaked. >> >> >> This same phenomenon occurs when I run a different UNIX program, e.g. wc: >> >> Output of wc -l Data/hs_chr3.fa: >> >> 2915465 Data/hs_chr3.fa >> >> Output of free: >> >> total used free shared buffers cached >> Mem: 513616 511520 2096 0 1252 481020 >> -/+ buffers/cache: 29248 484368 >> Swap: 128480 0 128480 >> >> Interestingly, after running wc a second time on the same file, it goes >> very fast and very little additional memory is leaked: >> >> total used free shared buffers cached >> Mem: 513616 510732 2884 0 1204 480948 >> -/+ buffers/cache: 28580 485036 >> Swap: 128480 40 128440 >> >> >> ------------------------------------------- >> Craig Cummings, Ph.D. >> >> Relman Laboratory >> Stanford University School of Medicine >> Department of Microbiology and Immunology >> >> e-mail: cummings@cmgm.stanford.edu >> phone: 650-498-5998 >> fax: 650-852-3291 >> >> - >> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> Please read the FAQ at http://www.tux.org/lkml/ End of included message -- +--------------------------+------------------------------+ | Tim Walberg | tewalberg@mediaone.net | | 828 Marshall Ct. | www.concentric.net/~twalberg | | Palatine, IL 60074 | | +--------------------------+------------------------------+