百度的蜘蛛果然是智能化的（续上篇百度蜘蛛思考）
接上篇[url]http://www.im286.com/thread-2755784-1-1.html[/url] 拯救千年发布站[url]www.1000350.com[/url] 被K实录看百度蜘蛛的智能程度！被K首页降权之后（原因已查明，半夜蜘蛛疯狂爬的时候被严重CC导致网站不能访问所致，查日志发现有大量连接站点动态文件）
百度是从被K当日12点后就没有再爬，站点改良是从下午4点的时候改良OK！并增加了几个原创文章
刚刚分析了一下日志，由于日志太大，其它的就不发了，只发百度爬过的：
2008-07-22 16:09:38 W3SVC2146169726 60.190.118.78 GET /open.asp id=3219 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 16:09:38 W3SVC2146169726 60.190.118.78 GET /head.asp id=3219 80 - 220.181.32.54 Baiduspider+2008-07-22 16:20:15 W3SVC2146169726 60.190.118.78 GET /img/bian.gif - 80 - 116.208.160.120 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+QQDownload+1.7) 304 0 0
2008-07-22 16:20:39 W3SVC2146169726 60.190.118.78 GET /yuanchuanggushi/index.html - 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 16:20:39 W3SVC2146169726 60.190.118.78 GET /yuanchuanggushi/index.html - 80 - 220.181.32.54 Baiduspider+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+QQDownload+1.7) 304 0 0
2008-07-22 16:30:40 W3SVC2146169726 60.190.118.78 GET /open.asp id=3216 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 16:30:40 W3SVC2146169726 60.190.118.78 GET /head.asp id=3216 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 16:40:56 W3SVC2146169726 60.190.118.78 GET /open.asp id=3218 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 16:40:56 W3SVC2146169726 60.190.118.78 GET /head.asp id=3218 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 16:43:48 W3SVC2146169726 60.190.118.78 HEAD /list/look_2172.html - 80 - 61.135.168.30 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 16:51:57 W3SVC2146169726 60.190.118.78 GET /list/look_3217.html - 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 17:01:59 W3SVC2146169726 60.190.118.78 GET /open.asp id=3213 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 17:01:59 W3SVC2146169726 60.190.118.78 GET /head.asp id=3213 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 17:12:00 W3SVC2146169726 60.190.118.78 GET /list/look_3219.html - 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 17:22:01 W3SVC2146169726 60.190.118.78 GET /head.asp id=3185 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 17:32:01 W3SVC2146169726 60.190.118.78 GET /head.asp id=3211 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 17:42:03 W3SVC2146169726 60.190.118.78 GET /head.asp id=3192 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 17:52:04 W3SVC2146169726 60.190.118.78 GET /head.asp id=3204 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:02:03 W3SVC2146169726 60.190.118.78 GET /open.asp id=3188 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:02:04 W3SVC2146169726 60.190.118.78 GET /head.asp id=3188 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:11:05 W3SVC2146169726 60.190.118.78 HEAD /index.html - 80 - 220.181.32.22 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:12:06 W3SVC2146169726 60.190.118.78 GET /open.asp id=3198 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:12:06 W3SVC2146169726 60.190.118.78 GET /head.asp id=3198 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:22:06 W3SVC2146169726 60.190.118.78 GET /open.asp id=3207 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:22:07 W3SVC2146169726 60.190.118.78 GET /head.asp id=3207 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:32:08 W3SVC2146169726 60.190.118.78 GET /open.asp id=3210 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 18:32:08 W3SVC2146169726 60.190.118.78 GET /head.asp id=3210 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 20:12:19 W3SVC2146169726 60.190.118.78 GET /head.asp id=3196 80 - 220.181.32.54 Baiduspider+2008-07-22 20:52:21 W3SVC2146169726 60.190.118.78 GET /list/look_2706.html - 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 21:02:23 W3SVC2146169726 60.190.118.78 GET /list/look_2527.html - 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 21:12:24 W3SVC2146169726 60.190.118.78 GET /list/look_2293.html - 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-07-22 21:32:26 W3SVC2146169726 60.190.118.78 GET /list/look_2536.html - 80 - 220.181.32.54 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
由此可见此站拯救几率为90%还要看继续努力的结果，百度已经开始爬刚刚发过的原创文章栏目首页！
估计下午还会继续爬！差不多这个站应该是会被重新收录主页了！
拯救指数：现在为三星！（待续，实在太困，不能再分析了，为了这个事已经熬了一夜了！）有需要帮助的做站兄弟可留下站点被K情况或QQ联系方式，本人尽量帮忙！觉得好的就顶下，觉得不好的别砸我了啊！
百度的蜘蛛是强大的，非常的智能化，老李的技术团队真的算得上NO.1
转载请注明出自落伍者论坛 [url]www.im286.com[/url] 站长的家
[[i] 本帖最后由 嘴βαbγ 于 2008-7-23 05:55 编辑 [/i]] 帮顶 ding :ohh: 学习了啊，不错的啊 顶。。 :lol: 分析的不错 不错--- 还没分析过这个 该站已经被重收，并上了第一页，HOHO~庆祝一下吧！ 曾经分析过 拯救指数：现在为三星，还有这一说，呵呵 支持!!! 不错,真被收了啊 新米百度都不来了~~~ 有被K了 我的站百度就不来~ 帮顶
来爬爬我的
[url]www.nowns.com[/url] 分析的不错啊 学习
[url=http://www.xm1rc.com]厦门人才[/url]
[url=http://www.591games.org.cn]彩虹岛[/url]
页:
[1]
2