VB.net 2010 视频教程 VB.net 2010 视频教程 python基础视频教程
SQL Server 2008 视频教程 c#入门经典教程 Visual Basic从门到精通视频教程
当前位置:
首页 > temp > 简明python教程 >

  • 第一个爬虫和测试(3)

　代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
from bs4 import BeautifulSoup
r = '''
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>菜鸟教程(runoob.com)  23号的作业</title>
</head>
<body>
 
<h1>我的第一个标题</h1>
<p id="first">我的第一个段落。</p>
 
</body>
    <table border="1">
        <tr>
            <td>row 1, cell 1</td>
            <td>row 1, cell 2</td>
        </tr>
    </table>
</html>
'''
demo = BeautifulSoup(r,"html.parser")
 
print(demo.title)
print(demo.body)
print(demo.p)
print(demo.string)

效果如下：

 

四、爬取中国大学排名（年费2016），将数据存为csv文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import requests
from bs4 import BeautifulSoup
ALL = []
def getHTMLtext(url):
    try:
        r = requests.get(url,timeout = 30)
        r.raise_for_status()
        r.encoding = 'utf-8'
        return r.text
    except:
        return ""
 
def fillUni(soup):
    data = soup.find_all('tr')
    for tr in data:
        td1 = tr.find_all('td')
        if len(td1) == 0:
            continue
        Single = []
        for td in td1:
            Single.append(td.string)
        ALL.append(Single)
 
def printUni(num):
    print("{1:^2}{2:{0}^10}{3:{0}^6}{4:{0}^6}{5:{0}^6}{6:{0}^6}{7:{0}^6}{8:{0}^6}{9:{0}^5}{10:{0}^6}{11:{0}^6}{12:{0}^6}{13:{0}^6}".format(chr(12288),"排名","学校名称","省市","总分",\
                                                               "生源质量","培养结果","科研规模","科研质量",\
                                                               "顶尖成果","顶尖人才","科技服务",\
                                                               "产学研究合作","成果转化"))
    for i in range(num):
          u = ALL[i]
          print("{1:^4}{2:{0}^10}{3:{0}^6}{4:{0}^8}{5:{0}^9}{6:{0}^9}{7:{0}^7}{8:{0}^9}{9:{0}^7}{10:{0}^9}{11:{0}^8}{12:{0}^9}{13:{0}^9}".format(chr(12288),u[0],\
                                                                                                                                                          u[1],u[2],eval(u[3]),\
                                                                                                                                                          u[4],u[5],u[6],u[7],u[8],\
                                                                                                                                                          u[9],u[10],u[11],u[12]))
 
def main(num):
          url = "http://www.zuihaodaxue.com/zuihaodaxuepaiming2016.html"
          html = getHTMLtext(url)
          soup = BeautifulSoup(html,"html.parser")
          fillUni(soup)
          printUni(num)
main(10)

相关教程