PDA

View Full Version : Transferring large data volume



ExtraSlow
10-10-2017, 08:24 AM
So I'm copying files from my old nas to my new one, and I'm wondering if there is an efficient method to do this.

The old drag and drop in windows explorer seems to be having issues.
Approximately 1TB in maybe 5000 files total.

Theres probably a better way, thoughts?

Thaco
10-10-2017, 08:44 AM
So I'm copying files from my old nas to my new one, and I'm wondering if there is an efficient method to do this.

The old drag and drop in windows explorer seems to be having issues.
Approximately 1TB in maybe 5000 files total.

Theres probably a better way, thoughts?

i use robocopy, then you can always pick up where you left off... there are lots of programs that will do it, but robocopy is free and does exactly what is needed.

ExtraSlow
10-10-2017, 09:14 AM
Thanks. I knew there had to be a system for this. Just grabbed richcopy as it seems like that's the updated version of robocopy.

Trying it now.

Alterac
10-10-2017, 09:18 AM
the problem.with a copy and paste from nas to nas in windows is that the file gets transferred through your computer to the other nas.

do either od your nas have a web interface with a file manager? maybe see if ou can copy nas to nas without the middleman.

depending in your technical level a ssh to the nas and rsync will solve this too.

e31
10-10-2017, 09:23 AM
the problem.with a copy and paste from nas to nas in windows is that the file gets transferred through your computer to the other nas.

do either od your nas have a web interface with a file manager? maybe see if ou can copy nas to nas without the middleman.

depending in your technical level a ssh to the nas and rsync will solve this too.

Most consumer NAS boxes have some web interface provision for scheduling a backup; just have the old NAS direct it to the desired directory on the new one, supply credentials, set Full backup to "now", Presto!

ExtraSlow
10-10-2017, 09:24 AM
So the computer in the middle is the bottleneck? That makes sense.

Alterac
10-10-2017, 09:42 AM
So the computer in the middle is the bottleneck? That makes sense.

well, it really depends on your nas's and how they are connected along with how your pc is connected.

if all a hardwired in with gigabit, and entry level nas devices, you most likely wont have a bottleneck with your pc.

but if your computer is on wireless it could get quite slow

ExtraSlow
10-10-2017, 09:45 AM
It's all hardwired, and the two NAS boxes are sitting side by side with just a switch in between. The desktop computer is also sitting right there and hardwired in through the same switch.

I do have some stuff to send over wireless eventually from one of my laptops, but for now everything is through copper.

Alterac
10-10-2017, 10:07 AM
assuming you have a dns-323 as your old nas, and the qnap ts-231p your new one, the biggest bottleneck is the dns-323 nas. it tops.out at around 15MB/s, and your computer and new nas can saturate a gigabit link (125MB/s)

i had a few of those dns-323 in the past before i changed to my new setup.

ExtraSlow
10-10-2017, 10:12 AM
Yeah that's my setup. The DNS323 wasn't working for a while, and now it is. Although it's possibly that it was just unplugged. Hahaha that's kind of embarrassing, but because both my wife and I were trying to troubleshoot thing thing, who knows who did what to it.

Having some issues to mapping drives as well. Win10 OS on the desktop. Win7 on the laptop. what a hassle.

Alterac
10-10-2017, 10:24 AM
are you mapping the drives with the IP of the nas, or the name?

if the name is failing try the ip, and double check the nas DNS settings.

pretty much your easy option is the robocopy and patience

revelations
10-10-2017, 12:05 PM
DNS323 uses an EXT2/3 (Linux) format. If all else fails, remove one drive (assuming RAID 1) and install in your PC with a EXT disk reader program. Then you can drag and drop using the robocopy program.

http://www.download3k.com/Install-DiskInternals-Linux-Reader.html

If all else fails, you can stop by with a bottle of booze here again and well get it done :rofl:

ExtraSlow
10-10-2017, 01:58 PM
That's a good option if this stupid transfer takes much longer.

Xtrema
10-10-2017, 02:38 PM
mount old location as E:
mount new location as F:

robocopy e:\ f:\ /s /e /copy:DAT /r:0 /log:c:\temp\copy.log

now you can check copy.log in c:\temp to see what failed to copy.

And if there is any disruptions, running the same command again we only bring over files that are newer in the old location. Also make sure both NAS and the windows machine is on the same switch and 1Gbps to maximize speed of transfer.

It shouldn't take you a few hrs to do this, if you have tons of small files, at worst 48hrs.

Thaco
10-10-2017, 02:47 PM
mount old location as E:
mount new location as F:

robocopy e:\ f:\ /s /e /copy:DAT /r:0 /log:c:\temp\copy.log

now you can check copy.log in c:\temp to see what failed to copy.

And if there is any disruptions, running the same command again we only bring over files that are newer in the old location. Also make sure both NAS and the windows machine is on the same switch and 1Gbps to maximize speed of transfer.

It shouldn't take you a few hrs to do this, if you have tons of small files, at worst 48hrs.
if you have to restart wouldn't you want to use /mir instead of /e to avoid re-doing everything you already did?

Xtrema
10-10-2017, 03:08 PM
/MIR do security as well, that's why I prefer /copy: DAT

Unless you are in a corporate environment, chances are you own everything. Although I have not run into any permission issue with any NAS these days, I still hate copying permissions unless it's needed.

Thaco
10-10-2017, 03:29 PM
/MIR do security as well, that's why I prefer /copy: DAT

Unless you are in a corporate environment, chances are you own everything. Although I have not run into any permission issue with any NAS these days, I still hate copying permissions unless it's needed.

makes sense, i only use mine in a corp environment so i guess thats why i always do /mir