落伍者站长论坛 » 站长知道(未落伍可发帖) » 怎么知道百度蜘蛛每天都来爬？

simpman 发表于 2008-6-24 01:23

我是挂个51啦统计，查看seo情况基本上能知道蜘蛛来了没。
没看过日志。:P

tiandi1900 发表于 2008-6-24 02:12

我所知道的都是有的!

eatice 发表于 2008-6-24 09:43

你可以用ftp登陆到你的空间,一般空间商都为你建立log文件夹的,
你可下载里面的txt日志回来看看,打开查找baidu就知道了

diycs 发表于 2008-6-24 10:27

在日志里面可以查得到的

xxcvb 发表于 2008-6-24 11:28

这里记录了蜘蛛的详细爬行过程

2008-06-13 06:34:35 W3SVC1424162025 220.194.55.171 GET /robots.txt - 80 - 67.195.37.187 Mozilla/5.0+(compatible;+Yahoo!+Slurp;+[url]http://help.yahoo.com/help/us/ysearch/slurp[/url]) 404 0 2
2008-06-13 06:34:37 W3SVC1424162025 220.194.55.171 GET /index.asp - 80 - 67.195.37.187 Mozilla/5.0+(compatible;+Yahoo!+Slurp/3.0;+[url]http://help.yahoo.com/help/us/ysearch/slurp[/url]) 200 0 0
2008-06-13 06:34:37 W3SVC1424162025 220.194.55.171 GET /Skin/DefaultSkin.css - 80 - 67.195.37.187 Mozilla/5.0+(compatible;+Yahoo!+Slurp/3.0;+[url]http://help.yahoo.com/help/us/ysearch/slurp[/url]) 200 0 0
2008-06-13 06:34:38 W3SVC1424162025 220.194.55.171 GET /news2.asp - 80 - 67.195.37.187 Mozilla/5.0+(compatible;+Yahoo!+Slurp/3.0;+[url]http://help.yahoo.com/help/us/ysearch/slurp[/url]) 200 0 0
2008-06-13 06:34:38 W3SVC1424162025 220.194.55.171 GET /news.asp - 80 - 67.195.37.187 Mozilla/5.0+(compatible;+Yahoo!+Slurp/3.0;+[url]http://help.yahoo.com/help/us/ysearch/slurp[/url]) 200 0 0
2008-06-13 06:34:40 W3SVC1424162025 220.194.55.171 GET /skin/css.css - 80 - 67.195.37.187 Mozilla/5.0+(compatible;+Yahoo!+Slurp/3.0;+[url]http://help.yahoo.com/help/us/ysearch/slurp[/url]) 200 0 0
2008-06-13 06:34:41 W3SVC1424162025 220.194.55.171 GET /skin/css.css - 80 - 67.195.37.187 Mozilla/5.0+(compatible;+Yahoo!+Slurp/3.0;+[url]http://help.yahoo.com/help/us/ysearch/slurp[/url]) 200 0 0
2008-06-13 06:34:49 W3SVC1424162025 220.194.55.171 GET /index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 64
2008-06-13 06:35:47 W3SVC1424162025 220.194.55.171 HEAD /xsyt/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:36:30 W3SVC1424162025 220.194.55.171 HEAD /tbtj/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:36:52 W3SVC1424162025 220.194.55.171 HEAD /scdy/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:37:06 W3SVC1424162025 220.194.55.171 HEAD /rctd/ShowArticle.asp ArticleID=8826 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:37:48 W3SVC1424162025 220.194.55.171 HEAD /rctd/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:37:53 W3SVC1424162025 220.194.55.171 HEAD /hygh/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:38:02 W3SVC1424162025 220.194.55.171 HEAD /gqxx/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:38:15 W3SVC1424162025 220.194.55.171 GET /gqxx/ShowHot.asp - 80 - 202.160.180.197 Mozilla/5.0+(compatible;+Yahoo!+Slurp+China;+[url]http://misc.yahoo.com.cn/help.html[/url]) 200 0 0
2008-06-13 06:38:17 W3SVC1424162025 220.194.55.171 HEAD /cyzx/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:38:40 W3SVC1424162025 220.194.55.171 HEAD /Article/index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0
2008-06-13 06:38:49 W3SVC1424162025 220.194.55.171 HEAD /index.asp - 80 - 61.135.168.112 Baiduspider+(+[url]http://www.baidu.com/search/spider.htm[/url]) 200 0 0

xxcvb 发表于 2008-6-24 11:30

2008-06-13 06:44:44 W3SVC1424162025 220.194.55.171 GET /cyzx/Comment.asp ArticleID=11815 80 - 203.208.60.33 Mozilla/5.0+(compatible;+Googlebot/2.1;++[url]http://www.google.com/bot.html[/url]) 200 0 0
  这是google他们家的蜘蛛

ncesun 发表于 2008-6-24 13:57

一般看logfiles下文件的记录，方便一点的话用逆火等软件看吧。

查看完整版本: 怎么知道百度蜘蛛每天都来爬？