博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
带断点续传的多线程下载
阅读量:6152 次
发布时间:2019-06-21

本文共 14168 字,大约阅读时间需要 47 分钟。

  • 多线程下载

原理:服务器CPU分配给每条线程的时间片相同,服务器带宽平均分配给每条线程,所以客户端开启的线程越多,就能抢占到更多的服务器资源,所以使用多线程下载的话,速度会更快

JavaSE实现带断点续传的多线程下载步骤:

  1、发送http请求至下载地址,获取要下载的资源文件的大小

  2、根据资源文件的大小,创建一个长度一样的临时文件,用来抢占磁盘空间

  3、计算每个线程要下载的数据大小和开始位置、结束位置,余数都由最后一个线程完成下载,所以最后一个线程的结束位置要写死

  4、再次发送请求,请求要下载的数据区间的数据(判断是否有记录进度的临时文件,有的话就继续上次位置接着下载,没有就从原本开始位置下载)

  5、将下载请求到的数据,存储到临时文件中(新建一个记录下载进度的临时文件)

  6、等所有线程都下载完毕了,就要将之前的记录进度的临时文件删除掉

1 package com.ahu.multithreaddownload;  2   3 import java.io.BufferedReader;  4 import java.io.File;  5 import java.io.FileInputStream;  6 import java.io.InputStream;  7 import java.io.InputStreamReader;  8 import java.io.RandomAccessFile;  9 import java.net.HttpURLConnection; 10 import java.net.URL; 11  12 /** 13  * 带断点续传的多线程下载 14  *  15  * @author ahu_lichang 16  *  17  */ 18 public class MultiThreadDownload { 19     // 线程数 20     static int ThreadCount = 4; 21     static int finishedThread = 0; 22  23     static String path = "http://172.23.13.179:8080/report.ppt"; 24  25     public static void main(String[] args) { 26         try { 27             // 第一次请求服务器,是为了获取资源文件的大小,从而创建一个相同大小的临时文件,而不是下载资源! 28             URL url = new URL(path); 29             HttpURLConnection connection = (HttpURLConnection) url 30                     .openConnection(); 31             connection.setConnectTimeout(5000); 32             connection.setReadTimeout(5000); 33             connection.setRequestMethod("GET"); 34             //connection.connect(); 35             if (connection.getResponseCode() == 200) { 36                 // 获取要下载文件的大小 37                 int length = connection.getContentLength(); 38                 // 生成临时文件 39                 RandomAccessFile raf = new RandomAccessFile(getFileName(path), 40                         "rwd"); 41                 raf.setLength(length); 42                 raf.close(); 43                 // 计算每个线程下载的资源大小(有可能存在余数,余数都放在最后一个线程里) 44                 int size = length / ThreadCount; 45                 // 计算每个线程下载的开始位置和结束位置 46                 for (int i = 0; i < ThreadCount; i++) { 47                     int startIndex = i * size; 48                     int endIndex = (i + 1) * size - 1; 49                     // 如果是最后一个线程,必须将结束位置写死 50                     if (i == ThreadCount - 1) { 51                         endIndex = length - 1; 52                     } 53                     // 开启线程下载资源 54                     new DownloadThread(startIndex, endIndex, i).start(); 55                 } 56             } 57         } catch (Exception e) { 58             e.printStackTrace(); 59         } 60  61     } 62  63     /** 64      * 获取服务器中资源的名称 65      *  66      * @param path 67      * @return 68      */ 69     public static String getFileName(String path) { 70         int index = path.lastIndexOf("/"); 71         return path.substring(index + 1); 72     } 73  74 } 75 /** 76  * 下载线程 77  * @author ahu_lichang 78  * 79  */ 80 class DownloadThread extends Thread { 81     int startIndex; 82     int endIndex; 83     int threadId; 84  85     public DownloadThread(int startIndex, int endIndex, int threadId) { 86         super(); 87         this.startIndex = startIndex; 88         this.endIndex = endIndex; 89         this.threadId = threadId; 90     } 91  92     public void run() { 93         try { 94             File progressFile = new File(threadId + ".txt"); 95             //如果存在进度临时文件,就接着上次的后面进行下载 96             if (progressFile.exists()) { 97                 FileInputStream fis = new FileInputStream(progressFile); 98                 BufferedReader br = new BufferedReader(new InputStreamReader( 99                         fis));100                 // 得到上一次下载的总进度,然后与原本的开始位置相加,得到新的开始位置101                 startIndex += Integer.parseInt(br.readLine());102                 fis.close();103             }104             System.out.println("线程" + threadId + "下载区间" + startIndex + "---"105                     + endIndex);106             // 再次请求服务器,下载资源文件107             URL url = new URL(MultiThreadDownload.path);108             HttpURLConnection connection = (HttpURLConnection) url109                     .openConnection();110             connection.setConnectTimeout(5000);111             connection.setReadTimeout(5000);112             connection.setRequestMethod("GET");113             // 设置本次所请求的数据区间114             connection.setRequestProperty("Range", "bytes=" + startIndex + "-"115                     + endIndex);116             //connection.connect();117             // 请求部分数据的响应码是206118             if (connection.getResponseCode() == 206) {119                 // 拿到1/3源文件的数据120                 InputStream is = connection.getInputStream();121                 byte[] b = new byte[1024];122                 int len = 0;123                 int total = 0;// 记录下载到临时文件中数据的大小124                 // 把拿到的数据写入到临时文件中125                 RandomAccessFile raf = new RandomAccessFile(126                         MultiThreadDownload127                                 .getFileName(MultiThreadDownload.path),128                         "rwd");129                 // 把文件的写入位置移动至startIndex。这样才不会覆盖写入130                 raf.seek(startIndex);131                 while ((len = is.read(b)) != -1) {132                     raf.write(b, 0, len);133                     total += len;134                     // 生成用来记录下载进度的临时文件135                     RandomAccessFile progressRaf = new RandomAccessFile(136                             progressFile, "rwd");137                     progressRaf.write((total + "").getBytes());138                     progressRaf.close();139                 }140                 System.out.println("线程" + threadId + "下载完毕!!!");141                 raf.close();142 143                 // 全部下载完成后,将临时存放进度的文件删除144                 MultiThreadDownload.finishedThread++;145                 // 注意线程安全问题146                 synchronized (MultiThreadDownload.path) {147                     if (MultiThreadDownload.finishedThread == MultiThreadDownload.ThreadCount) {148                         for (int i = 0; i < MultiThreadDownload.ThreadCount; i++) {149                             File f = new File(i + ".txt");150                             f.delete();151                         }152                         MultiThreadDownload.finishedThread = 0;153                     }154                 }155             }156         } catch (Exception e) {157             e.printStackTrace();158         }159     }160 }
View Code

 

  • Android上实现带断点续传的多线程下载

布局文件:

1 
2
13
18 19
24
29 30
View Code

   MainActivity:

1 package com.ahu.lichang.multithreaddownload;  2   3 import android.app.Activity;  4 import android.os.Bundle;  5 import android.os.Environment;  6 import android.os.Handler;  7 import android.view.View;  8 import android.widget.ProgressBar;  9 import android.widget.TextView; 10  11 import java.io.BufferedReader; 12 import java.io.File; 13 import java.io.FileInputStream; 14 import java.io.InputStream; 15 import java.io.InputStreamReader; 16 import java.io.RandomAccessFile; 17 import java.net.HttpURLConnection; 18 import java.net.URL; 19  20 public class MainActivity extends Activity { 21     static int ThreadCount = 3; 22     static int finishedThread = 0; 23  24     int currentProgress; 25     String fileName = "QQPlayer.exe"; 26     //确定下载地址 27     String path = "http://172.23.13.179:8080/" + fileName; 28     private ProgressBar pb; 29     TextView tv; 30  31     Handler handler = new Handler(){ 32         public void handleMessage(android.os.Message msg) { 33             //把变量改成long,在long下运算 34             tv.setText((long)pb.getProgress() * 100 / pb.getMax() + "%"); 35         } 36     }; 37     @Override 38     protected void onCreate(Bundle savedInstanceState) { 39         super.onCreate(savedInstanceState); 40         setContentView(R.layout.activity_main); 41         pb = (ProgressBar) findViewById(R.id.pb); 42         tv = (TextView) findViewById(R.id.tv); 43     } 44     public void download(View view){ 45         Thread t = new Thread(){ 46             @Override 47             public void run() { 48                 //发送get请求,请求这个地址的资源 49                 try { 50                     URL url = new URL(path); 51                     HttpURLConnection conn = (HttpURLConnection) url.openConnection(); 52                     conn.setRequestMethod("GET"); 53                     conn.setConnectTimeout(5000); 54                     conn.setReadTimeout(5000); 55  56                     if(conn.getResponseCode() == 200){ 57                         //拿到所请求资源文件的长度 58                         int length = conn.getContentLength(); 59  60                         //设置进度条的最大值就是原文件的总长度 61                         pb.setMax(length); 62  63                         File file = new File(Environment.getExternalStorageDirectory(), fileName); 64                         //生成临时文件 65                         RandomAccessFile raf = new RandomAccessFile(file, "rwd"); 66                         //设置临时文件的大小 67                         raf.setLength(length); 68                         raf.close(); 69                         //计算出每个线程应该下载多少字节 70                         int size = length / ThreadCount; 71                         for (int i = 0; i < ThreadCount; i++) { 72                             //计算线程下载的开始位置和结束位置 73                             int startIndex = i * size; 74                             int endIndex = (i + 1) * size - 1; 75                             //如果是最后一个线程,那么结束位置写死 76                             if(i == ThreadCount - 1){ 77                                 endIndex = length - 1; 78                             } 79                             new DownLoadThread(startIndex, endIndex, i).start(); 80                         } 81                     } 82                 } catch (Exception e) { 83                     e.printStackTrace(); 84                 } 85             } 86         }; 87         t.start(); 88     } 89     class DownLoadThread extends Thread{ 90         int startIndex; 91         int endIndex; 92         int threadId; 93  94         public DownLoadThread(int startIndex, int endIndex, int threadId) { 95             super(); 96             this.startIndex = startIndex; 97             this.endIndex = endIndex; 98             this.threadId = threadId; 99         }100 101         @Override102         public void run() {103             //再次发送http请求,下载原文件104             try {105                 File progressFile = new File(Environment.getExternalStorageDirectory(), threadId + ".txt");106                 //判断进度临时文件是否存在107                 if(progressFile.exists()){108                     FileInputStream fis = new FileInputStream(progressFile);109                     BufferedReader br = new BufferedReader(new InputStreamReader(fis));110                     //从进度临时文件中读取出上一次下载的总进度,然后与原本的开始位置相加,得到新的开始位置111                     int lastProgress = Integer.parseInt(br.readLine());112                     startIndex += lastProgress;113 114                     //把上次下载的进度显示至进度条115                     currentProgress += lastProgress;116                     pb.setProgress(currentProgress);117                     //发送消息,让主线程刷新文本进度118                     handler.sendEmptyMessage(1);119 120                     fis.close();121                 }122                 System.out.println("线程" + threadId + "的下载区间是:" + startIndex + "---" + endIndex);123                 HttpURLConnection conn;124                 URL url = new URL(path);125                 conn = (HttpURLConnection) url.openConnection();126                 conn.setRequestMethod("GET");127                 conn.setConnectTimeout(5000);128                 conn.setReadTimeout(5000);129                 //设置本次http请求所请求的数据的区间130                 conn.setRequestProperty("Range", "bytes=" + startIndex + "-" + endIndex);131 132                 //请求部分数据,相应码是206133                 if(conn.getResponseCode() == 206){134                     //流里此时只有1/3原文件的数据135                     InputStream is = conn.getInputStream();136                     byte[] b = new byte[1024];137                     int len = 0;138                     int total = 0;139                     //拿到临时文件的输出流140                     File file = new File(Environment.getExternalStorageDirectory(), fileName);141                     RandomAccessFile raf = new RandomAccessFile(file, "rwd");142                     //把文件的写入位置移动至startIndex143                     raf.seek(startIndex);144                     while((len = is.read(b)) != -1){145                         //每次读取流里数据之后,同步把数据写入临时文件146                         raf.write(b, 0, len);147                         total += len;148                         System.out.println("线程" + threadId + "下载了" + total);149 150                         //每次读取流里数据之后,把本次读取的数据的长度显示至进度条151                         currentProgress += len;152                         pb.setProgress(currentProgress);153                         //发送消息,让主线程刷新文本进度154                         handler.sendEmptyMessage(1);155 156                         //生成一个专门用来记录下载进度的临时文件157                         RandomAccessFile progressRaf = new RandomAccessFile(progressFile, "rwd");158                         //每次读取流里数据之后,同步把当前线程下载的总进度写入进度临时文件中159                         progressRaf.write((total + "").getBytes());160                         progressRaf.close();161                     }162                     System.out.println("线程" + threadId + "下载完毕");163                     raf.close();164 165                     finishedThread++;166                     synchronized (path) {167                         if(finishedThread == ThreadCount){168                             for (int i = 0; i < ThreadCount; i++) {169                                 File f = new File(Environment.getExternalStorageDirectory(), i + ".txt");170                                 f.delete();171                             }172                             finishedThread = 0;173                         }174                     }175                 }176             } catch (Exception e) {177                 e.printStackTrace();178             }179         }180     }181 }
View Code
1     
2
3

 

 

 

转载于:https://www.cnblogs.com/ahu-lichang/p/6594009.html

你可能感兴趣的文章
批量删除oracle中以相同类型字母开头的表
查看>>
Java基础学习总结(4)——对象转型
查看>>
BZOJ3239Discrete Logging——BSGS
查看>>
SpringMVC权限管理
查看>>
spring 整合 redis 配置
查看>>
cacti分组发飞信模块开发
查看>>
浅析LUA中游戏脚本语言之魔兽世界
查看>>
飞翔的秘密
查看>>
Red Hat 安装源包出错 Package xxx.rpm is not signed
查看>>
编译安装mysql-5.6.16.tar.gz
查看>>
活在当下
查看>>
每天进步一点----- MediaPlayer
查看>>
PowerDesigner中CDM和PDM如何定义外键关系
查看>>
跨域-学习笔记
查看>>
the assignment of reading paper
查看>>
android apk 逆向中常用工具一览
查看>>
MyEclipse 报错 Errors running builder 'JavaScript Validator' on project......
查看>>
Skip List——跳表,一个高效的索引技术
查看>>
Yii2单元测试初探
查看>>
五、字典
查看>>