大家好，又见面了，我是你们的朋友全栈君。
下载目标是堆糖网热门图片，打开网页并下拉发现图片是通过ajax加载的，按F12打开开发者工具选择nerwork并筛选xhr，继续下拉网页找到ajax请求的api，如下图所示
![堆糖网热门图片下载[通俗易懂]](https://img2020.cnblogs.com/blog/1448761/202006/1448761-20200604193638655-616754313.png)
![堆糖网热门图片下载[通俗易懂]](https://javaforall.net/wp-content/uploads/2020/11/2020110817443450.jpg)
然后就可以构造请求获取包含图片url的json数据，对于网络请求等IO密集型任务，开启进程池可以提高下载速度
代码如下：
import requests from requests import exceptions import re from multiprocessing import Pool import os def get_pic_info(): url = 'https://www.duitang.com/napi/index/hot/?' for i in range(1000): params = { 'include_fields': 'top_comments,is_root,source_link,item,buyable,root_id,status,like_count,sender,album', 'limit': '24', 'start': 24 * i, } response = requests.get(url, params=params) json_data = response.json() pic_list = json_data['data']['object_list'] for pic_ in pic_list: image = {} pic_info = pic_['album'] pic_url = pic_info['covers'][0] image['pic_name'] = re.sub(r'[\\/:*?"<>|\r\n。，.？ ]+', '', pic_info['name']) + '.' + pic_url.split('.')[-1] image['pic_url'] = pic_url yield image def download_pic(image): if not os.path.exists(f'./img/{image["pic_name"]}'): try: resp = requests.get(image['pic_url']) if resp.status_code == 200: with open(f'./img/{image["pic_name"]}', 'wb') as f: f.write(resp.content) except exceptions: return None else: print(image['pic_name'] + ' has already downloaded') if __name__ == '__main__': if not os.path.exists('./img'): os.mkdir('./img') pool = Pool() pool.map(download_pic, get_pic_info()) pool.close() pool.join()
版权声明：本文内容由互联网用户自发贡献，该文观点仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请联系我们举报，一经查实，本站将立刻删除。
发布者：全栈程序员-站长，转载请注明出处：https://javaforall.net/155277.html原文链接：https://javaforall.net
