Created
October 1, 2018 14:08
-
-
Save 798886844/49366e52a7dac4e8f014ba3ecc2ce2b9 to your computer and use it in GitHub Desktop.
VPS+Rclone实现网盘对拷
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
网盘的稳定性一直是网盘用户的痛点。 | |
360倒闭之后许多文件来不及下载,而后又没有合适的地方存储,导致很多人咬牙掏钱买硬盘。然而更多没钱的人选择了其他网盘,比如我,最后选择了OnedriveForBusiness。 | |
但是无论数据存储在哪里,都有丢失的时候,存硬盘里面硬盘会坏,存Onedrive里面……额,订阅哪来的你们也清楚,所以做好数据备份很重要。 | |
所以我一直在寻找这样一个方法,能够帮我完成Onedrive For Business网盘对拷功能。一开始尝试了Flow,然而不知道怎么回事,最开始还能用,到后面就不行了,估计是用的人太多了吧。然后又选择了MultCloud,然而这货有2T流量限制,速度还死慢,作为没钱的我,只好选择了第三条路,利用手头的资源自己搭建。 | |
经过不懈的努力摸索,今晚洗澡的时候我终于萌发出灵感:利用Rclone和VPS进行网盘对拷。下面我具体来说说吧: | |
搭建环境:Docker环境,镜像:[rastasheep/ubuntu-sshd](https://hub.docker.com/r/rastasheep/ubuntu-sshd/) | |
其中,该镜像的环境是 | |
```c# | |
ubuntu-sshd-910292714-kpnx7 4.4.0-72-generic | |
用户名、密码均为:root | |
``` | |
需要开放的端口包括:22、80、6800,但不限于以上端口。 | |
好了,接下来开始操作: | |
### Reclone安装与挂载 | |
关于Reclone挂在网盘的操作方法,我主要参考了以下两篇文章: | |
* [可乐博客-通过Rclone及Oneindex实现Onedrive离线下载教程](https://www.cokemine.com/rclone-onedrive-oneindex.html) | |
* | |
* [Rat'sBlog-在Debian/Ubuntu上使用rclone挂载OneDrive网盘](https://www.moerats.com/archives/491/) | |
有问题的可以去原帖看一下。 | |
下面开始说我的操作: | |
首先,需要一台Windows电脑: | |
打开[https://rclone.org/downloads/](https://rclone.org/downloads/),下载Windows-AMD64 - 64 Bit压缩包文件,并解压至`D:\rclone`。 | |
然后打开管理员命令提示符,输入 | |
```C# | |
cd /d d:\rclone | |
rclone authorize "onedrive" | |
``` | |
登陆你的账户,获取code,例如我的: | |
```language | |
{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayIsImtpZCI6ImlCakwxUmNxemhpeTRmcHhJeGRacW9oTTJZayJ9.eyJhdWQiOiJodHRwczovL2FwaS5vZmZpY2UuY29tL2Rpc2NvdmVyeS8iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9kNWEzODc3YS02ZjBkLTQ4NzUtOWUyMS03MjZhMWJkOTRjMTUvIiwiaWF0IjoxNTI3MzIwOTczLCJuYmYiOjE1MjczMjA5NzMsImV4cCI6MTUyNzMyNDg3MywiYWNyIjoiMSIsImFpbyI6IlkyZGdZRmgrdjYxRFRIcm5xcm1OOVpIbFNxWDNsRU92ekZrMjd3ZXIzTXprMUdXK0JxOEEiLCJhbXIiOlsicHdkIl0sImFwcGlkIjoiNTI4NTdmZWMtNGJjMi00ODNmLTlmMWItNWZlMjhlOTc1MzJjIiwiYXBwaWRhY3IiOiIxIiwiaXBhZGRyIjoiMTEwLjE4NC4xNzkuMTMwIiwibmFtZSI6InRlc3QiLCJvaWQiOiI2YTQzMGIzOC01N2VhLTQxZTQtYWI0My1iZDhiZTZmNzUwZmIiLCJwdWlkIjoiMTAwMzNGRkZBQjlDMEQxRSIsInNjcCI6Ik15RmlsZXMuUmVhZCBNeUZpbGVzLldyaXRlIFVzZXIuUmVhZCIsInN1YiI6IjlseWt4dlBNUGNmT2FOWVlqQUNkbDNyaUs1YkY1anFUS0RmY003SDg5QjAiLCJ0aWQiOiJkNWEzODc3YS02ZjBkLTQ4NzUtOWUyMS03MjZhMWJkOTRjMTUiLCJ1bmlxdWVfbmFtZSI6InRlc3RAZ2hkZ2pkZi5vbm1pY3Jvc29mdC5jb20iLCJ1cG4iOiJ0ZXN0QGdoZGdqZGYub25taWNyb3NvZnQuY29tIiwidXRpIjoiV28yOE1pWGpRa1doTGQ0RGdKd0VBQSIsInZlciI6IjEuMCJ9.QtuYv9vpSYZSNNlRDLYlf0MjLo399aWz26PIVX7lz9FWceeCkydGjMtrWhb7S5qgy2Z-yqnJnN-XpZAdlztM7x5CSqbPNSX1n95YNxnW6Bl423rdosIw6xjDkBQf7jFvPutlGgVkNEmzk-znmJwSDNtsEY8j5LUzLrtWpNweAWK-I35TT5hb6ejXP66NsJc3yDP7-Qy3fhUX402vZl4NZCMkb1G4DVm6dNoWihxcZWhrP91eYqfMy_XlaZhexyD0NUqV2O28y3Rs_aGn2MMJonTBOdj18vV31DBiTooD-ImA3W69gDqThH85Ql8eRoMZ1j76hwELLPKPA7-UjCmkBg","token_type":"Bearer","refresh_token":"AQABAAAAAADX8GCi6Js6SK82TsD2Pb7rnd9hYhHoPmWiPciFQK-L_jFPvodZQ9cadNF6AYwGPH73A-rW3fUWBZ6ZIRNzWeqXotbtGERnsbsY9oSv2dESZf0PAZ0SmlhliTB55p-UHMBhhA02RZzqRO-VYQxC6PYMP20P8PBSvqS7BvkurelSog8L4EpncgyQWpof16aHz6mVgJ3CD-OA7m9zmF3k41TSNhoYxeDznfe4eUYe0vHwbz7Ubfv7xpNwiXKw2JuqkL_UikS-Nmq3_ls-suRZ0VDQxRT7uEc_Z-0H4PeIVH2Cvai7EoQ2WVfZXFZzV8qS_ryjbWspnO6u2_WyReEDUIukHR7BILUOVi141i1L6VXEpw6S-x4TGqKf5q88-BPtbmNz9IwO7nwIVkGe5vY111yVYbQg3j3JzbW0eReByKnSETyypovMEEk_08TYJdD-Ks7AByAUolDrZhFuEUDZlh4VGNNG5DGTf4opoaSNYNt4Rkf9R2b1rL46x66DOBGD2cRMtSUAuy1pNSt1YdddYwV4tiiY15XzCdGiY8Hm39gw9T3eVP1ja_SaQRCiKpZfr63BUO3Hzt1DITgmYqUC_NtDzcyG3c_lbnUrCl4QB2h5f37U8cdRrHJ0uzWwezFL80xmLCdA1Xl-V-OEKjqvWvBNuitNuVbX_20V7wTQgQWSXcwOU1I_BYOwI1aCUmjQBax2GGJQDSVAPu4jJVQOG-7FV8S2__5-ushbSsCqVmqi14ECKQPsm_4pwuDx9W8Tphpbw68fVgSHn3eJqAunovydIAA","expiry":"2018-05-26T16:54:32.8494162+08:00"} | |
``` | |
这一段保存好,可以反复使用。 | |
然后打开shell工具,连接你的VPS,开始挂载: | |
安装相关软件: | |
```language | |
apt-get update | |
apt-get install -y unzip fuse vim | |
``` | |
安装Reclone: | |
```c# | |
wget https://downloads.rclone.org/rclone-current-linux-amd64.zip | |
unzip rclone-current-linux-amd64.zip | |
chmod 0777 ./rclone-*/rclone | |
cp ./rclone-*/rclone /usr/bin/ | |
rm -rf ./rclone-* | |
``` | |
配置Reclone: | |
`rclone config` | |
然后出现以下内容: | |
```c# | |
2018/05/26 08:09:11 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults | |
No remotes found - make a new one | |
n) New remote | |
s) Set configuration password | |
q) Quit config | |
n/s/q> n #选择"New remote" | |
name> OneDrive #自定义网盘名称 | |
Type of storage to configure. | |
Choose a number from below, or type in your own value | |
1 / Alias for a existing remote | |
\ "alias" | |
2 / Amazon Drive | |
\ "amazon cloud drive" | |
3 / Amazon S3 Compliant Storage Providers (AWS, Ceph, Dreamhost, IBM COS, Minio) | |
\ "s3" | |
4 / Backblaze B2 | |
\ "b2" | |
5 / Box | |
\ "box" | |
6 / Cache a remote | |
\ "cache" | |
7 / Dropbox | |
\ "dropbox" | |
8 / Encrypt/Decrypt a remote | |
\ "crypt" | |
9 / FTP Connection | |
\ "ftp" | |
10 / Google Cloud Storage (this is not Google Drive) | |
\ "google cloud storage" | |
11 / Google Drive | |
\ "drive" | |
12 / Hubic | |
\ "hubic" | |
13 / Local Disk | |
\ "local" | |
14 / Mega | |
\ "mega" | |
15 / Microsoft Azure Blob Storage | |
\ "azureblob" | |
16 / Microsoft OneDrive | |
\ "onedrive" | |
17 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH) | |
\ "swift" | |
18 / Pcloud | |
\ "pcloud" | |
19 / QingCloud Object Storage | |
\ "qingstor" | |
20 / SSH/SFTP Connection | |
\ "sftp" | |
21 / Webdav | |
\ "webdav" | |
22 / Yandex Disk | |
\ "yandex" | |
23 / http Connection | |
\ "http" | |
Storage> 16 #选择网盘类型,这里选择16:Onedrive | |
Microsoft App Client Id - leave blank normally. | |
client_id> #留空 | |
Microsoft App Client Secret - leave blank normally. | |
client_secret> #留空 | |
Remote config | |
Choose OneDrive account type? | |
* Say b for a OneDrive business account | |
* Say p for a personal OneDrive account | |
b) Business | |
p) Personal | |
b/p> b #我是教育版,属于Onedrive For Business,故选择“b) Business” | |
Use auto config? | |
* Say Y if not sure | |
* Say N if you are working on a remote or headless machine | |
y) Yes | |
n) No | |
y/n> n #因为是远程操作,所以选择“n) No” | |
For this to work, you will need rclone available on a machine that has a web browser available. | |
Execute the following on your machine: | |
rclone authorize "onedrive" | |
Then paste the result below: | |
result> #粘贴上面那一大串code,然后会提示下面的数据请你确认 | |
-------------------- | |
[OneDrive] | |
type = onedrive | |
client_id = | |
client_secret = | |
token = | |
resource_url = | |
-------------------- | |
y) Yes this is OK | |
e) Edit this remote | |
d) Delete this remote | |
y/e/d> y 确认后选择 "y" | |
Current remotes: | |
Name Type | |
==== ==== | |
OneDrive onedrive | |
e) Edit existing remote | |
n) New remote | |
d) Delete remote | |
r) Rename remote | |
c) Copy remote | |
s) Set configuration password | |
q) Quit config | |
e/n/d/r/c/s/q> q #如果需要连接第二个网盘,选择“n”,如果不需要添加其他网盘了,选择“q”退出。 | |
``` | |
重复上述步骤,再连接一个网盘。 | |
到这里,已经完成了网盘在该VPS上的登录认证,并已经映射为一个叫做OneDrive的网盘。 | |
完成上面对两个OFB网盘的连接后,开始对拷: | |
键入命令`rclone copy gdrive1: gdrive2:` | |
表示从gdrive1的根目录开始将所有文件拷贝到gdrive2,同时gdrive2里面的数据可能会被完全清空。 | |
给大家看一下效果: | |
首先是我在对拷的过程中,用宝塔面板进行了后台服务性能的监控,大概峰值就在这个样子: | |
 | |
可以看到其实对于CPU的使用并没有像病毒一样的疯狂占满,只在有的时候可能会稍微大一点,所以建议用空的小鸡对拷。另外可以看到,对本机的空间占用几乎没有,所以无论大鸡小鸡都能做到。 | |
在看一下接口流量: | |
 | |
可以看出,速度受下载和上传两个因素同时限制。 | |
再看一下十分钟的拷贝速度和效果: | |
 | |
 | |
需要提醒的是,因为即使是对拷,用的也是服务器本机的流量,包括下载和上传的流量。所以如果你想拷50G的文件,就要花费至少100G的流量,这是需要计费的。 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
看到你的方法解决了我的freenas上jail 无法验证token的问题,我也分享个小技巧同样类型的网盘可以实现 sever-side copy 不走本地流量rclone copy "remote1:소스" "remote2:타겟" --drive-server-side-across-configs -P -v,不过跨网盘还是要本地流量哈,这是视频https://www.youtube.com/watch?v=3MRWENXyEb4&list=PLTanR3CoBkXphIx1vA-OHPrnHwqd3ZKvP&index=2