VB.net 2010 视频教程 VB.net 2010 视频教程 python基础视频教程
SQL Server 2008 视频教程 c#入门经典教程 Visual Basic从门到精通视频教程
当前位置:
首页 > 编程开发 > C#教程 >
  • 记一次 .NET 某电商定向爬虫 内存碎片化分析

一:背景

1. 讲故事

上个月有位朋友wx找到我,说他的程序存在内存泄漏问题,寻求如何解决? 如下图所示:

从截图中可以看出,这位朋友对 windbg 的操作还是有些熟悉的,可能缺乏一定的实操经验,所以用了几个命令之后就不知道怎么排查下去了。

既然找到我,那就以我的个人经验在他的dump上继续分析寻找罪魁祸首,闲话不多说,上windbg说话。

二:Windbg 分析

1. 真的存在内存泄漏吗?

追这个系列的朋友应该知道,我无数次的用 !address -summary 和 !eeheap -gc 这两个命令来判断当前的内存泄漏是属于托管层还是非托管层。


0:000> !address -summary

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                    358     7dfc`67f60000 ( 125.986 TB)           98.43%
<unknown>                              1087      203`88b6e000 (   2.014 TB)  99.99%    1.57%
Image                                  1532        0`09f11000 ( 159.066 MB)   0.01%    0.00%
Heap                                    249        0`03453000 (  52.324 MB)   0.00%    0.00%
Stack                                    66        0`01fc0000 (  31.750 MB)   0.00%    0.00%
Other                                    10        0`001d1000 (   1.816 MB)   0.00%    0.00%
TEB                                      22        0`0002c000 ( 176.000 kB)   0.00%    0.00%
PEB                                       1        0`00001000 (   4.000 kB)   0.00%    0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_MAPPED                              183      200`00d06000 (   2.000 TB)  99.30%    1.56%
MEM_PRIVATE                            1252        3`8d479000 (  14.207 GB)   0.69%    0.01%
MEM_IMAGE                              1532        0`09f11000 ( 159.066 MB)   0.01%    0.00%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_FREE                                358     7dfc`67f60000 ( 125.986 TB)           98.43%
MEM_RESERVE                             749      200`28a9b000 (   2.001 TB)  99.33%    1.56%
MEM_COMMIT                             2218        3`6f5f5000 (  13.740 GB)   0.67%    0.01%

0:000> !eeheap -gc
Number of GC Heaps: 1
generation 0 starts at 0x00000026DA8DA928
generation 1 starts at 0x00000026DA7FC348
generation 2 starts at 0x00000024C4691000
ephemeral segment allocation context: none
         segment             begin         allocated              size
00000024C4690000  00000024C4691000  00000024D468FF28  0xfffef28(268431144)
00000024EECF0000  00000024EECF1000  00000024FECF0000  0xffff000(268431360)
000000248D6F0000  000000248D6F1000  000000249D6EFEF8  0xfffeef8(268431096)
...
00000026D66D0000  00000026D66D1000  00000026DBA3CA30  0x536ba30(87472688)
Large object heap starts at 0x00000024D4691000
         segment             begin         allocated              size
00000024D4690000  00000024D4691000  00000024DC67C318  0x7feb318(134132504)
00000024E60F0000  00000024E60F1000  00000024EE0637C8  0x7f727c8(133638088)
0000002482140000  0000002482141000  000000248A08F338  0x7f4e338(133489464)
00000024A6770000  00000024A6771000  00000024AE76F6C0  0x7ffe6c0(134211264)
...
000000278E6D0000  000000278E6D1000  000000279635F2D0  0x7c8e2d0(130605776)
00000029233E0000  00000029233E1000  000000292AF672F8  0x7b862f8(129524472)
000000292B3E0000  000000292B3E1000  0000002931A5ED60  0x667dd60(107470176)
000000299B3E0000  000000299B3E1000  00000029A20095B0  0x6c285b0(113411504)
000000281E6D0000  000000281E6D1000  0000002825CD3F58  0x7602f58(123744088)
00000028266D0000  00000028266D1000  000000282D5CAD50  0x6ef9d50(116366672)
000000282E6D0000  000000282E6D1000  0000002833CA0880  0x55cf880(89979008)
00000029A33E0000  00000029A33E1000  00000029A684D300  0x346c300(54969088)
Total Size:              Size: 0x353f96d88 (14293757320) bytes.
------------------------------
GC Heap Size:    Size: 0x353f96d88 (14293757320) bytes.

从输出看,当前进程占用 MEM_COMMIT=13.7G,托管堆内存占用 14293757320 = 13.3G,很明显这属于简单模式的 托管内存泄漏,根据经验,托管堆上可能有什么大对象,这里用 !dumpheap -stat 命令。


0:000> !dumpheap -stat
Statistics:
              MT    Count    TotalSize Class Name
00007ff9ed6ea268  3956842     94964208 System.Collections.Generic.Dictionary`2+KeyCollection[[System.String, System.Private.CoreLib],[Serilog.Events.LogEventPropertyValue, Serilog]]
00007ff9ed5e6d28  3842435    166405016 Serilog.Parsing.MessageTemplateToken[]
00007ff9ed5e5e28  3842434    184436832 Serilog.Events.MessageTemplate
00007ff9ecccf090  4011012    203304420 System.Int32[]
00007ff9ed647078  3956849    253238336 Serilog.Events.LogEvent
00007ff9ed6e7b48  3956849    284893128 System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib],[Serilog.Events.LogEventPropertyValue, Serilog]]
00007ff9ed5e74e8  9259598    296307136 Serilog.Parsing.TextToken
00007ff9ed6471b0 12551808    301243392 Serilog.Events.ScalarValue
00007ff9ed6e8308  3956849    729078048 System.Collections.Generic.Dictionary`2+Entry[[System.String, System.Private.CoreLib],[Serilog.Events.LogEventPropertyValue, Serilog]][]
00007ff9eccb1e18 16546412   3987811940 System.String
00000024c3b8faf0    82904   7382993568      Free

我去,托管堆最大的对象居然是 Free,大概占用 7.3G, 这就


相关教程