图文实例——微博美食博主搜索列表页面信息采集

2 讨 论 1046 浏览量 云采集引擎

微博搜索列表页采集涉及到两个问题:一个是用户登录,第二个就是如何获取一个列表页的多条信息

本实例学习可以结合数据源: 微博搜索美食博主信息采集

数据源创建步骤如下:

第一步,配置基本信息

第二步,入口配置

这里的入口其实就是我们搜索结果的链接

打开微博登陆后查询关键字,然后获取浏览器中的链接

可以看到最后的数字随着页数的增加而递增,所以这里使用自动增长配置

最后的页数部分使用[page]通配符替换,然后在下面设置抓取的范围(这里我抓取了第一页到第五十页的内容)

第三步,链接配置

采集列表页内容和一般采集详情页面配置方法不同。

是通过连接配置获取所有需要采集的关键内容,然后作参数传递到字段配置,通过脚本正则分割获取数据

所以连接配置步骤如下:

首先通过url识别规则获取每个博主的信息内容

然后由于获取的内容不是正常的连接而是数据,这里需要补充一个连接作为前缀访问

一般可以直接设置百度,不会封IP,记得在最后加上一个“#”分割数据和连接

然后大家可以看到上面获取的数据是个unicode编码需要转换,这里可以直接使用现成的自定义脚本:自定义脚本-unicode编码转换

这样连接配置完成之后,列表页采集到的数据就会传递到字段配置

第四步,字段配置

这一步我们要分割获取到的每个字段的信息,比如名称,id,地址,描述等等

首先使用[currenturl]获取到传递过来的所有数据

然后通过自定义脚本的正则处理获取数据

所有字段都通过这个方法获取

最后一步,设置cookie信息

最开始我们说了采集微博搜索列表页有两个问题,一个是列表页多条信息的采集上面已经解决了

第二个就是登陆的问题。

这里需要回到微博搜索的界面,打开开发者选项(F12)

找到我们采集的连接,然后选中,在右边找到头信息中的cookie值

将整个复制出来黏贴到我们的高级功能中,这样云采集就能模拟你的登录信息了

——END——

2讨 论
已讨论 6月 21日 作者: ojvtracax
<a href="https://aldactonefast.com/">aldactone potassium-sparing</a>
<a href=https://aldactonefast.com/>aldactone 50 mg tablet </a>
https://aldactonefast.com - spironolactone 50 mg
<a href="https://allegrax.com/">fexofenadine hydrochloride</a>
<a href=https://allegrax.com/>allegra d </a>
https://allegrax.com - fexofenadine 180 mg
<a href="https://fastcymbalta.com/">cymbalta side effects</a>
<a href=https://fastcymbalta.com/>cymbalta </a>
https://fastcymbalta.com - side effects for cymbalta
已讨论 7月 22日 作者: onnyyrer
One might be able to predict when sexual activity will occur.The only proven way to get Versed out of your system is to stop taking it and allow your body time to process and eliminate it. https://cialismans.com/ - cialis pill Your blood pressure could drop to an unsafe level are allergic to sildenafil, as contained in VIAGRA and REVATIO, or any of the ingredients in VIAGRA.With Viagra about to reach its 20th birthday, there s no question that this little blue pill has already had a big impact.- Albina Janabayeva did not stop it.From the nervous system increased excitability of muscles, trembling of limbs, decreased attention, increased nervous excitability. <a href=https://webmedicxxx.com/>viagra generic</a> My dick was so hard and throbbing so hard I thought it was going to split wide the fuck open.prenominal Syn nonprescription prenominal.Randy couple Saffron and Leo tested Cialis, which is dubbed the weekend pill , which allows men to perform for up to 36-hours. https://superpillsx.com/ - is tramadol a controlled substance Intended for women with hypoactive sexual desire disorder HSDD , it is thought to work on certain chemicals in the brain serotonin, dopamine and norepinephrine to increase sexual motivation.Overdose With a single dose of the drug Viagra at a dose of up to 800 mg undesirable phenomena were the same as when taking the drug in lower doses, but were more common.Prescribed for Erectile Dysfunction, Pulmonary Hypertension. <a href=https://goodshoprx.com/>discount viagra</a> The FDA has warned that such supplements, not being prescription medication, have yet to prove their efficacy and safety.The pros of ordering the tablets online involve the following. <a href=https://wowcialisnow.com/>online cialis</a> Also, having suffered a stroke or heart attack in the last six months is not recommended to take sialis.It is used for the treatment of hypertension, congestive heart failure, acute myocardial infarction, and diabetic nephropathy.To prepare the natural remedy, you need to pour 50 grams of fresh leaves of double-dom nettles with a glass of boiling water.STUDENT A s QUESTIONS Do not show these to student B. <a href="https://wowcialisnow.com/">buy cialis online safely</a> But taking into account the physiological features of the body, it is recommended to perform sexual intercourse no later than 5 hours after taking the medicine.You can either download it onto your mobile, or print it to show your doctor or pharmacist.Its components do not accumulate in the blood. <a href=https://wowcialisnow.com/>cialis 5mg</a> It works wonders.over the counter over the count er adj 1 not traded or effected on an organized securities exchange over the counter transactions over the counter securities 2 sold lawfully without a prescription over the counter drugs Merriam Webster s Dictionary of Law Law dictionary.Some people congratulated Ms. <a href=https://wowcialisnow.com/>cialis 2.5 mg</a>

Finndy Copyright©2017 | Powered by Q2A

...