We are using robocopy with MIR switch to backup our Microsoft Access Databases, i.e., .accdb, and .mdb files. Interesting thing with Access database files is that you can open a database, add a few records into a table, close the db and the size of the database file will not change.
Will robocopy know that he has to copy this file next time when he does the backup?

Thanks

Changing the file not necessarily leads to a change in the file’s size, but typically (i.e. unless a sufficiently privileged process has changed that) the following always happens:

• the “last modified” timestamp of the file is updated to the current timestamp
• the “archive” bit is touched (if it was not set, it is set by the OS automatically)

Robocopy would compare the “last modified” timestamp of the destination with that of the source and copy when these two differ (this behavior might be tweaked according to your requirements using the /xn and /xo switches). Robocopy would only honor (and reset) the archive file attribute when you use /m switch – see the robocopy documentation for details.

Check more discussion of this question.

Is it possible to download a file from the web using RoboCopy? What would the syntax for that be? Here’s what I’m trying:

RoboCopy "http://www.google.com/" "C:\some_folder\" "index.html" /L /V /E /LOG:"C:\some_folder\test_robocopy.log" /R:10 /W:30

The error log is giving me an error that seems to indicate the file is a folder, and also that it’s trying to hit http://www.google.com as a folder on my location machine.

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------  Started : Tue Jan 08 10:31:04 2013   Source : C:\some_folder\http:\www.google.com\index.html\
     Dest : C:\some_folder\index.html\    Files : *.*  Options : *.* /V /L /S /E /COPY:DAT /R:10 /W:30 ------------------------------------------------------------------------------2013/01/08 10:31:04 ERROR 123 (0x0000007B) Accessing Source Directory C:\some_folder\http:\www.google.com\index.html\
The filename, directory name, or volume label syntax is incorrect.

The only way I can conceive of making this work is if you create a WebDAV share, such as the one at live.sysinternals.com, and then address it as a UNC (\\yoursite.com\folder) and the minidirector in the WebClient service will handle it.

But that requires a specific configuration on the web server.

Also, Invoke-WebRequest if you’re using PS 3.

Edit: Also make sure the WebClient service is running before trying to access WebDAV shares on the command line. Windows Explorer will start the service on demand, but cmd.exe or powershell.exe will not.

Check more discussion of this question.

I’ve got an old win 2003 VM (VMware) on a blade cluster of VMs that I’m moving a considerable amount of files to our new DFS array. There are two main folders with about 1.7 million and half a million smaller files (letters, memos, and other smaller files) respectively. Total size is ~420 GB and ~100 GB.

We’re using the gui version of robocopy on the server to copy the files. We had initiated a file copy about a month ago to test the process and found that it was taking around 4 hours for the large file. Now that I’m in the process of actually switching the files over it has been taking 18-20 hours. Nothing has changed on the server side and nothing has changed on the settings of the copy (no logs, 1 retry with a wait of 1 second).

Our intent is to shut off the share and force the copy over again to get all the files that have been left out of the copy due to being locked by users. I can’t take a 20 hour outage to do that though.

Does anyone have any theories about what could be causing such a delay for robocopy compared to previously shorter runs?

Edit: I discovered last night the copy must be freezing. It stops at a certain time (seems to be variable) and doesn’t continue at all.

We’ve been fiddling with this one. Looks like there was something odd going on with the gui interface. When I specifically set the retry and wait to “1″ it went through normally. This may have been coincidence, but it began working normally on the first try with those settings.

Check more discussion of this question.

We are using Robocopy to redistribute files which arrive via FTP in a drop folder. Ideally we want Robocopy to leave files alone if they are still being FTPd.

Having tried various switches, Robocopy still copies the open files. It doesn’t delete them, so the FTP continues unaffected. However, we end up with truncated files being distributed to their destination.

Can Robocopy be made to skip open files?

Perhaps there is something more suitable than Robocopy for this task?

Have you tried using the /R and /W switches?

Something like:

/R:2 /W:2 

This means retry twice wait 2 secs to retry. If the file is still open after the two attempts, the file will be skipped. Depending on your needs you could lower it to /R:1 /W:1.

Check more discussion of this question.

The Problem

We would like to backup our critical files from several network shares to a removable hard drive. We want to automate the backup so we don’t have to remember to run it. It needs to finish overnight. Furthermore, we want to be able to preserve multiple versions of each file so we can back out of our user’s mistakes easier.

Background Information

I work in a large Windows-based enterprise with a centralized IT section who is responsible for all backups. Their backups are geared towards disaster recovery and not user error, and require upper-level management approval for any non-disaster recoveries. Several times we have noticed that our backups have failed, we weren’t notified. I do not have administrative rights to the server or my desktop. We are trying to backup some 198,000 files spanning about 240 gigabytes. These files rarely change. Our backup drive is one terabyte.

My Proposed Solution

What I would like to do is to write a batch file using Robocopy with the /mir option along with Mercurial SCM to store all versions of the file. I would do an hg add followed by an hg commit before each execution of Robocopy to save the current state, and then make a mirrored copy of the file structures. The problem is the /mir will delete every folder not present in the source, and Mercurial stores the repository in a .hg folder in the destination folder.

Does anybody know how I could either convince Mercurial to store the .hg folder elsewhere, or convince Robocopy not to delete it from the destination?

I’m trying to avoid writing a custom program do to copying.

To perform a mirror while ignoring certain file exentions, try this:

robocopy c:\temp\source c:\temp\dest /e /purge /xf *.hg

I tested it a bit here and it appears to leave the .hg files alone on the destination

I also recommend the /z flag (restartable for large files) and have had to resort to the /fft flag in the past when dealing with servers having different time granularity : http://www.readynas.com/forum/viewtopic.php?t=3573

edit:
… my example above was assuming files named *.hg … to do the same and ignore directories named *.hg, use this :

robocopy c:\temp\source c:\temp\dest /e /z /purge /xd *.hg

Check more discussion of this question.

I have the the following setup.

1 x host machine situated in DOMAIN A
1 x PDC in a VM running on a host only network adapter running its own AD Forest (DOMAIN B)
1 x Server running in DOMAIN B, this is also on a host only network adapter but has a secondary adapter which is running in NAT mode Via VirtualBox (the PDC is running in the same VirtualBox instance).

I need to take files in DOMAIN A and copy using ROBOCOPY to DOMAIN B where no trust relationship exists, DOMAIN B is totally isolated and is never allowed to talk to DOMAIN A in any fashion at all.

One way of doing this would be to use PSEXEC from sysinternals and copy the files using the networkpath to the files FROM DOMAIN B this approach has been ruled out entirely, for various reasons, one of which is time and the reworking of a process that would take too long (allthough it would be possible)

I can copy the files with runas with the /netonly switch without issue manually from a cmd line giving run as the password at runtime, this all works.

The problem is I am running this copy from inside a rake command which invokes Robocopy using the MSBuild Extensions (just executes Robocopy.exe).

Is there a thirdparty of runas or equivalent tool that allows the /netonly behaviour of runas with the password given as a switch ?

the equivalent syntax for runas (if runas supported this behaviour) would be the following:

runas /user:user@domain.com /password:password /netonly "c:\system32\robocopy.exe C:\somedir \"\\TARGETSERVER\c$\somedir\" \"*.*\" /MIR /A-:R"

You could simply connect to the destination share through a different security context using the net use \\targetserver\c$\somedir /USER:DOMAIN\user password syntax.

Omitting the drive letter in the net use call creates an authenticated SMB connection to a remote share within the given security context of DOMAIN\user. After the connection is established, you can use references to \targetserver\c$\somedir in your robocopy call.

Check more discussion of this question.

I have a series of scripts setup to backup my Exchange. The following command is executed to start the process:

diskshadow /s C:\Backup_Scripts\exchangeserverbackupscript1.dsh

This is exchangeserverbackupscript1.dsh:

#DiskShadow script fileset verbose on
#delete shadows all
set context persistent
writer verify {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
set metadata C:\Backup_Scripts\shadowmetadata.cab
begin backup
add volume C: alias SH1createexpose %SH1% P:
exec C:\Backup_Scripts\exchangeserverbackupscript1.cmd
end backup
delete shadows exposed P:
exit
#End of script

And this is exchangeserverbackupscript1.cmd:

robocopy "P:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group" "\\leahyfs\J$\E-Mail Backups\Day 1" /MIR /R:0 /W:0 /COPY:DT /B

This is not causing Exchange to purge its log files. The edb file is 4.7 gigabytes, but the First Storage Group folder itself is 50+ gigabytes due to many, many log files for each day going back to 2009.

Is there any way — I’ve Googled and haven’t found anything — to notify Exchange when I’ve completed a full backup, and have it purge its log files?

According to this and this, end backup should cause Exchange to “flush the transaction logs for that storage group” but only “if a successful backup of a storage group occurred“, which leaves my question as:

What constitutes a “successful backup”, and why is what I’m doing not it?

In general, the BEGIN BACKUP / END BACKUP block would signal Exchange to truncate the logs. But in order for it to work, you should have created a snapshot of all volumes with Exchange-related data – so if you have separated data and transaction logs, you would need to snapshot both – the data and the Tlog volume.

Also, you should make sure that your script exchangeserverbackupscript1.cmd really returns with an exit code of zero – if it does not the diskshadow script would abort and the “end backup” line would never get executed. I believe robocopy would exit with the exit code of “1″ upon successful file copy which might present the root of your problem as the errorlevel of robocopy would be the errorlevel of your batch if the robocopy is the last executed command. Try adding a conditional checking for robocopy’s success and an exit /B 0 to the end of your batch file.

Check more discussion of this question.

in our small sized company we use Vmware. We backup the VM with Veeam.
Resulting files are in order of 200-300 gigabyte per VM, for a total of 1/1.2 terabyte of data.
I use robocopy to copy VM files backuped to external hard drive. I would like to know if Robocopy use something like rsync to copy only changed ‘blocks’ of data instead the entirely file .
Thanks

Robocopy doesn’t use any method to transmit only changed blocks of datas.

If you have to transmit your files thru a slow network, just use rsync.
If you have to transmit your files thru a “reasonable speed” network, no algorithm can speedup the process, because rsync (for example) have to read the entire file on client and on server to find duplicate contents. So if your network is Gigabit, or if you do a local copy, you just copy the files and voila. You can’t get something better, execpted if supported by the filesystem (ala ZFS, but only works on the same partition).

Check more discussion of this question.

I’m copying a file over to another directory and was wondering if Robocopy can change the modified date to the date it copied the file over in the new directory. I haven’t been successful playing with the switches.

Set Source="C:\Dir\
Set Dest="C:\Dir\TMProbocopy /V /IS "%Source%" "%Dest%" /NOCOPY

I want test.txt (modified 01/23/12) to copy over to the TMP directory with a new date modified of time of copy.

My solution was to use a batch file:

Echo %date% %time% > testfile.txt

That will include the date and time within the text file and also update the modified date to the file.

Check more discussion of this question.

I have a web site generating files at runtime, this web site is deployed on 2 servers

Operation system, Windows Web Server 2008 R2.

I am looking for a manner to sync the changed and new files from one server to another, one-way sync.

DFS-R, anybody succeed in installing DFS-R on Windows Web Server 2008 R2? here is an article http://blogs.technet.com/b/filecab/archive/2008/05/14/installing-dfs-replication-on-windows-server-2008.aspx, which does not work for 2008 R2 any more.

robocopy, I tried this one, but it does not perform very well and sometimes there is everal minutes delay before the changed file is synced.

Please do you have any suggestion to me?

Thanks

I use DFS-R on a bunch of Windows 2008 R2 Standard Webservers for the web content without a problem. They all need to be members of a domain, but I remember the wizard being fairly clickety-straightforward.

You simply install the DFS components on all the servers you want to sync, and then on one of them set up all the replicated folders and add all the clients to that replicated folder.

This link seems to cover the steps quite well.

N.B. – you say ‘realtime’ – this means different things to different people. What sort of a lag is tolerable for you? Solutions vary (in complexity and cost) depending on whether you can tolerate 5 minutes, 5 seconds, or 5 milliseconds.

Edit

I missed where you said you had Server 2008 R2 Web Edition, you’re right that doesn’t come with DFS. This leaves you with two other options:

1. Save all runtime configures to some persistent shared storage, either a network drive that all nodes can see, or to a database somehow.

2. Hack together Robocopy, rsync, or some other filesystem sync framework to handle the replication.

Personally, I’ve never found (2) to be particularly reliable. It might work at first, but silent failures can cause a lot of issues. If it’s only small files, (1) might not be a bad idea. In an ideally architected world, web servers shouldn’t be storing transient but shared data on their filesystems anyway, precisely because of this synchronization issue. However, I appreciate that re-building everything just to solve this might not be the most available option. In this case, I’d probably push for resource to upgrade to Server 2008 Standard Edition – DFS really does work very well, and MS presumably removed it from Web Edition to push more advanced deployments onto Standard.

Alternatively, you can take a look at the IIS Web Farm Framework. I’ve never played with it myself, but it looks like it might have some filesystem syncrhonization features in it. I can’t guarantee that it’ll do what you want though.

Check more discussion of this question.