Fix memory leak
We are running a FreeNAS 11.1-STABLE, the box has 144G memory, while the ARC was at 80G, and not increasing. This seems to be the cause of nearly all memory wired, and not released to ARC.
The box had around 137G in wired state, around 4G free, and had just 80G in ARC. Running a very simple perl script, which allocates 4G memory and writes to it, resulted immeditately in 14G free memory, reduced wired memory, and with the same ARC size. After half an hour, the ARC began to increase, with wired memory also, but I suppose 86G arc memory should not need 128G wired memory.
Now this is the top header:
Mem: 91M Active, 283M Inact, 1527M Laundry, 128G Wired, 11G Free
ARC: 86G Total, 23G MFU, 55G MRU, 287M Anon, 1796M Header, 6787M Other
67G Compressed, 146G Uncompressed, 2.16:1 Ratio
Swap: 10G Total, 52M Used, 10G Free
I would better see not used wired memory as free, as in that case I am sure the ARC would be able to grow. Please find the ARC size graph from our nas, the decrease can be observed.
It could hurt performance, I suspect.
#6 Updated by Richard Kojedzinszky almost 2 years ago
On #27235 I've mentioned there may be a memory leak in g_bio uma zone, as we see huge numbers as used.
Please investigate. This can be reproduced easily just booting up a box, and issuing IO to a pool. In a few minutes, I got 200 entries persistently in used state, while on other FreeBSD boxes, used counts are nearly at zero.
I used kernel based on commit 881ac279c45b2d5673607a25b0b8a45fcbb876d0.
#9 Updated by Alexander Motin almost 2 years ago
- Subject changed from ARC and wired memory to ARC and g_bio zone memory leak
- Status changed from Unscreened to Fix In Progress
- Priority changed from No priority to Blocks Until Resolved
- Target version set to 11.1-U1
Looking on FreeBSD ZFS commit log, there is a commit r323796, fixing g_bio zone memory leak in ZFS, triggered exactly by the ZFS commit you mentioned. It looks very promising. It should be in next hightly build due to massive FreeBSD merge, and I am going to merge it for 11.1-U1.