大文件上傳下載實現(xiàn)思路，分片、斷點續(xù)傳代碼實現(xiàn)
回復架構師獲取資源
大家好，我是你們的朋友架構君，一個會寫代碼吟詩的架構師。
'javajgs.com';
原文：https://blog.csdn.net/weixin_52210557
大文件上傳
實現(xiàn)思路：
分片：按照自定義緩沖區(qū)大小，將大文件分成多個小文件片段。
斷點續(xù)傳：根據(jù)分片數(shù)量，給每個小文件通過循環(huán)起對應名稱，當文件下載中斷在續(xù)傳時，判斷小文件名稱若存在則不存了，此時還需要判斷文件若不是最后一個分片則大小為緩沖區(qū)固定大小，若沒達到則證明小文件沒傳完需要重新傳輸。
合并：下載時通過線程池創(chuàng)建任務進行下載或上傳、當判斷最后一個分片傳完時，調(diào)用合并方法，根據(jù)之前定義的文件名稱順序進行合并，肯能出現(xiàn)最后一個分片傳完，之前分片未傳完的情況，需要使用while循環(huán)進行判斷，多文件未傳輸完，則等待一會繼續(xù)判斷。
大文件秒傳：實際上是根據(jù)文件名稱區(qū)一個唯一的md5值存儲，傳文件時進行判斷，若存在則不傳。
創(chuàng)建springboot項目，添加依賴
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<dependency>
<groupId>commons-fileupload</groupId>
<artifactId>commons-fileupload</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<!-- 做斷點下載使用-->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
編寫測試環(huán)境看環(huán)境有沒有搭建成功
@Controller
public class UploadController {
@RequestMapping("/up")
@ResponseBody
public String upload(HttpServletRequest request, HttpServletResponse response){
return "搭建成功";
}
}
頁面主要代碼
<body>
<div id="upload-container">
<span>上傳</span>
</div>
<div id="upload-list"></div>
<button id="picker">點擊上傳</button>
</body>
<script>
$('#upload-container').click(function (event){
$("#picker").find('input').click();
});
var uploader = WebUploader.create({
auto: true,
swf : 'Uploader.swf', //swf文件路徑
server: 'http://localhost:8080/upload',
dnd: '#upload-container',
pick: '#picker', //內(nèi)部根據(jù)當前運行創(chuàng)建
multiple: true, //選擇多個
chunked: true, //開啟分片
threads: 20, //并發(fā)數(shù)
method: 'POST',
fileSizeLimit: 1024*1024*1024*10, //單個文件大小限制
fileSingleSizeLimit: 1024*1024*1024, //總文件大小
fileVal: 'upload'
});
uploader.on("beforeFileQueued",function (file){
console.log(file); //獲取文件后綴
});
uploader.on('fileQueued',function (file){
//選中文件要做的事
console.log(file.ext);
console.log(file.size);
console.log(file.name);
var html = '<div class="upload-item"><span>文件名：'+file.name+'</span><span data-file_id="'+file.id+'"class="btn-delete">刪除</span><span data-file_id="'+file.id+'"class="btn-retry">重試</span><div class="percentage '+file.id+'" style="width: 0%;"></div></div>'
$('#upload-list').append(html);
uploader.md5File(file) //給文件定義唯一的md5值，當再次上傳相同文件時，就不用傳了 大文件秒傳實際上是沒傳，直接拷貝之前文件地址
//顯示進度
.progress(function (percentage){
console.log('Percentage:',percentage);
})
//完成
.then(function (val){
console.log('md5 result',val);
});
});
webUpload組件支持分片上傳：利用多進程并發(fā)上傳，將大文件拆分成一個一個的小文件，每一個小文件屬于大文件的一個分片
斷點續(xù)傳實現(xiàn)：后端代碼
@Controller
public class UploadController {
private final static String utf8 = "utf-8";
@RequestMapping("/up")
@ResponseBody
public void upload(HttpServletRequest request, HttpServletResponse response) throws Exception {
response.setCharacterEncoding(utf8);
//長傳時候會有多個分片，需要記錄當前為那個分片
Integer schunk = null;
//總分片數(shù)
Integer schunks = null;
//名字
String name = null;
//文件目錄
String path = "D:\\file";
BufferedOutputStream os = null;
try {
//設置緩沖區(qū)大小 先讀到內(nèi)存里在從內(nèi)存寫
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setSizeThreshold(1024);
factory.setRepository(new File(path));
//解析
ServletFileUpload upload = new ServletFileUpload(factory);
//設置單個大小與最大大小
upload.setFileSizeMax(5l*1024l*1024l*1024l);
upload.setSizeMax(10l*1024l*1024l*1024l);
List<FileItem> items = upload.parseRequest(request);
for (FileItem item : items){
if (item.isFormField()){
//獲取分片數(shù)賦值給遍量
if ("chunk".equals(item.getFieldName())){
schunk = Integer.parseInt(item.getString(utf8));
}
if ("chunks".equals(item.getFieldName())){
schunks = Integer.parseInt(item.getString(utf8));
}
if ("name".equals(item.getFieldName())){
name = item.getString(utf8);
}
}
}
//取出文件基本信息后
for (FileItem item : items){
if (!item.isFormField()){
//有分片需要臨時目錄
String temFileName = name;
if (name != null){
if (schunk != null){
temFileName = schunk+"_"+name;
}
//判斷文件是否存在
File temfile = new File(path, temFileName);
//斷點續(xù)傳 判斷文件是否存在，若存在則不傳
if (!temfile.exists()){
item.write(temfile);
}
}
}
}
//文件合并 當前分片為最后一個就合并
if (schunk != null && schunk.intValue()== schunks.intValue()-1){
File tempFile = new File(path, name);
os = new BufferedOutputStream(new FileOutputStream(tempFile));
//根據(jù)之前命名規(guī)則找到所有分片
for (int i = 0; i < schunks; i++) {
File file = new File(path, i + "_" + name);
//并發(fā)情況 需要判斷所有 因為可能最后一個分片傳完，之前有的還沒傳完
while (!file.exists()){
//不存在休眠100毫秒后在從新判斷
Thread.sleep(100);
}
//分片存在 讀入數(shù)組中
byte[] bytes = FileUtils.readFileToByteArray(file);
os.write(bytes);
os.flush();
file.delete();
}
os.flush();
}
response.getWriter().write("上傳成功");
}finally {
try {
if (os != null){
os.close();
}
}catch (IOException e){
e.printStackTrace();
}
}
}
}
文件分片下載服務端
@Controller
public class DownLoadController {
private final static String utf8 = "utf-8";
@RequestMapping("/down")
public void downLoadFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
response.setCharacterEncoding(utf8);
//定義文件路徑
File file = new File("D:\\File\\a.mp4");
InputStream is = null;
OutputStream os = null;
try {
//分片下載
long fSize = file.length();//獲取長度
response.setContentType("application/x-download");
String fileName = URLEncoder.encode(file.getName(),utf8);
response.addHeader("Content-Disposition","attachment;filename="+fileName);
//根據(jù)前端傳來的Range 判斷支不支持分片下載
response.setHeader("Accept-Range","bytes");
//獲取文件大小
response.setHeader("fSize",String.valueOf(fSize));
response.setHeader("fName",fileName);
//定義斷點
long pos = 0,last = fSize-1,sum = 0;
//判斷前端需不需要分片下載
if (null != request.getHeader("Range")){
response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
String numRange = request.getHeader("Range").replaceAll("bytes=","");
String[] strRange = numRange.split("-");
if (strRange.length == 2){
pos = Long.parseLong(strRange[0].trim());
last = Long.parseLong(strRange[1].trim());
//若結束字節(jié)超出文件大小 取文件大小
if (last>fSize-1){
last = fSize-1;
}
}else {
//若只給一個長度 開始位置一直到結束
pos = Long.parseLong(numRange.replaceAll("-","").trim());
}
}
long rangeLenght = last-pos+1;
String contentRange = new StringBuffer("bytes").append(pos).append("-").append(last).append("/").append(fSize).toString();
response.setHeader("Content-Range",contentRange);
response.setHeader("Content-Lenght",String.valueOf(rangeLenght));
os = new BufferedOutputStream(response.getOutputStream());
is = new BufferedInputStream(new FileInputStream(file));
is.skip(pos);//跳過已讀的文件
byte[] buffer = new byte[1024];
int lenght = 0;
//相等證明讀完
while (sum < rangeLenght){
lenght = is.read(buffer,0, (rangeLenght-sum)<=buffer.length? (int) (rangeLenght - sum) :buffer.length);
sum = sum+lenght;
os.write(buffer,0,lenght);
}
System.out.println("下載完成");
}finally {
if (is!= null){
is.close();
}
if (os!=null){
os.close();
}
}
}
}
客戶端分片下載，指定固定文件
@RestController
public class DownloadClient {
private final static long per_page = 1024l*1024l*50l;
//分片存儲臨時目錄 當分片下載完后在目錄中找到文件合并
private final static String down_path="D:\\File";
//多線程下載
ExecutorService pool = Executors.newFixedThreadPool(10);
//文件大小 分片數(shù)量 文件名稱
//使用探測 獲取變量
//使用多線程分片下載
//最后一個分片下載完 開始合并
@RequestMapping("/downloadFile")
public String downloadFile() throws IOException {
FileInfo fileInfo = download(0,10,-1,null);
if (fileInfo!= null){
long pages = fileInfo.fSize/per_page;
for (int i = 0; i <= pages; i++) {
pool.submit(new Download(i*per_page,(i+1)*per_page-1,i,fileInfo.fName));
}
}
return "成功";
}
class Download implements Runnable{
long start;
long end;
long page;
String fName;
public Download(long start, long end, long page, String fName) {
this.start = start;
this.end = end;
this.page = page;
this.fName = fName;
}
@Override
public void run() {
try {
FileInfo fileInfo = download(start,end,page,fName);
} catch (IOException e) {
e.printStackTrace();
}
}
}
//返回文件名 跟大小
private FileInfo download(long start,long end,long page,String fName) throws IOException {
//斷點下載 文件存在不需要下載
File file = new File(down_path, page + "-" + fName);
//探測必須放行 若下載分片只下載一半就鍛煉需要重新下載所以需要判斷文件是否完整
if (file.exists()&&page != -1&&file.length()==per_page){
return null;
}
//需要知道 開始-結束 = 分片大小
HttpClient client = HttpClients.createDefault();
//httpclient進行請求
HttpGet httpGet = new HttpGet("http://127.0.0.1:8080/down");
//告訴服務端做分片下載
httpGet.setHeader("Range","bytes="+start+"-"+end);
HttpResponse response = client.execute(httpGet);
String fSize = response.getFirstHeader("fSize").getValue();
fName= URLDecoder.decode(response.getFirstHeader("fName").getValue(),"utf-8");
HttpEntity entity = response.getEntity();//獲取文件流對象
InputStream is = entity.getContent();
//臨時存儲分片文件
FileOutputStream fos = new FileOutputStream(file);
byte[] buffer = new byte[1024];//定義緩沖區(qū)
int ch;
while ((ch = is.read(buffer)) != -1){
fos.write(buffer,0,ch);
}
is.close();
fos.flush();
fos.close();
//判斷是不是最后一個分片
if (end-Long.valueOf(fSize)>0){
//合并
try {
mergeFile(fName,page);
} catch (Exception e) {
e.printStackTrace();
}
}
return new FileInfo(Long.valueOf(fSize),fName);
}
private void mergeFile(String fName, long page) throws Exception {
//歸并文件位置
File file = new File(down_path, fName);
BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(file));
for (int i = 0; i <= page; i++) {
File tempFile = new File(down_path, i + "-" + fName);
//分片沒下載或者沒下載完需要等待
while (!file.exists()||(i!=page&&tempFile.length()<per_page)){
Thread.sleep(100);
}
byte[] bytes = FileUtils.readFileToByteArray(tempFile);
os.write(bytes);
os.flush();
tempFile.delete();
}
File file1 = new File(down_path, -1 + "-null");
file1.delete();
os.flush();
os.close();
}
//使用內(nèi)部類實現(xiàn)
class FileInfo{
long fSize;
String fName;
public FileInfo(long fSize, String fName) {
this.fSize = fSize;
this.fName = fName;
}
}
}
這些年小編給你分享過的干貨
2.優(yōu)質(zhì)ERP系統(tǒng)帶進銷存財務生產(chǎn)功能（附源碼）
3.優(yōu)質(zhì)SpringBoot帶工作流管理項目（附源碼）
5.SBoot+Vue外賣系統(tǒng)前后端都有（附源碼）
轉(zhuǎn)發(fā)在看就是最大的支持??
評論
圖片
表情