Folder Syncronization Concepts

Who is Folder Sync For? Folder sync has two primary uses:

Website maintenance - Normally you have on your desktop a copy of your website that you edit. Whenever you think you done with your changes for now, you want to publish to your website. This is extremely easy with AASync, the sync engine will automatically find your changes and publish your new and changed files only to the remote server. You may sync as often as you like.
Backups - Make copies of your important data to remote servers or to others disks on the desktop system. Either way it is the same easy-to-use interface!

Folder Sync means a very simple thing: You have a source folder whose contents you edit. Whenever the sync operation is performed, the destination folder will get the deltas only, so changed files are copied to the destination folder. This sounds very simple, but first we have to clear the air about the different kinds of Folder Syncronizations that are possible. Here is a quick explanation of terms:

Source Folder: The folder whose contents you change (like your local copy of your website). When you are done changing it, you run a Local-to-Remote sync to publish your changes.

Destination(Target) Folder: The folder that you do not touch directly, the sync engine will publish your changes from the Source Folder into the Destination Folder. This destination folder can reside on your local machine or on a remote host on which you have an FTP or SSH/SFTP account.

Phase 1, Sync by Source Database: In this phase, the destination folder is not looked at at all, the source folder's state is compared to the source folder's previous state. The new state is then stored in a Database file in the source folder under the name .rbtp_sync_db. This method has various advantages: the speed first of all, since there is no need for network access. Secondly the accuracy: we do not have to put up with the differences between various systems, comparing file names checksums, mod dates are relatively messy across the internet. The source Database comparison is very accurate, it can easily compare checksums and all kinds of meta data. The remote destination folder is then accessed just to upload the changed files. The question is, is this enough? In most cases, yes, if we are reasonably certain that the destination folder is there, it is more less in sync, and it is not being altered by somebody else! Also, this methos is not available for RTL sync for obvious reasons.

Phase 2, Sync by Source-Destination Comparison: In this phase, the destination folder is accessed by some protocol, like FTP/SFTP/SSH, then source and destination files are actually compared by the most accurate means possible. FTP is a poor protocol, but SSH actually allows a very accurate comparison, since checksums of the remote files are available on most systems. This phase is more time-consuming then Phase 1, but, when it completes, one can be reasonably sure that source and destination are in sync.

How does it work?

AASync automagically finds the changed files using various methods (Phase 1 and 2 described above), then recursively goes through the source folder and transfers all the changed files and ONLY the changed files. Files are always transferred whole.

FTP(with SSL too) or SFTP-SSH can be used in remote sync operations.

Here is the Sync Panel:

The columns:

The sync name: name your sync to anything you like, this column is editable
The sync status: OK, Error or Warning.. the status of the last sync.
Schedule modes:
- - the sync will start on a schedule.
- - once this sync finishes it will run another. See the Scheduling Panel to see which one.
- - after the sync has been performed on schedule, it will attempt to hibernate the host computer.
- - after the sync has been performed on schedule, it will attempt to suspend the host computer into standby mode.
The last sync time, making it very convenient to check when this sync was performed last. If you want to see the log of the last run just click the "Last Log" icon under the menus.

Scheduled Sync by Service:

If AASync's sync service is installed, that means you only have to launch the GUI version to manage the syncs, read logs etc. The service will run the scheduled syncs as long as the machine is turned on. If the machine is in standby/hibernate mode, AASyncService will attempt to wake it up 20 seconds before a scheduled run.

Things to know about Remote-To-Local Folder (RTL) Sync

There are a number of limitations to RTL sync: FTP as a protocol is simply too limited to support RTL sync. This is due to the fact that most FTP servers reset modification dates when a file is transferred to it, checksum is not available and size alone simply cannot detect change to a file. Therefore, RTL sync is only avalable in using the SFTP protocol. RTL sync is a simple one phase operation, both remote source and local destination are scanned once. The other limitation is that if a file is simply renamed in the source, RTL sync still has to treat it like a new file, so it is downloaded as if it were new. This is due to the fact that a rename cannot be detected deterministically just by reading remote data, since you can have more files with identical contents.

Things to know about Local-to-Remote (LTR) Folder Sync

The sync engine in AASync employs a two-phase sync system for Local-to-Remote Folder Sync. In the first phase, it builds a database in a hidden file that is put in the source folder. This database records the state of the source and compares it to the previous database file. This way, changes can be detected easily without having to rely on various protocols to get information from the remote system. This is especially important if FTP is the protocol used to communicate with the remote system, since FTP is such a poor protocol, that it really does not allow to deterministically track changes. It trashes mod dates, does not have checksum and with text files even the file sizes change during uploads. Not to mention the fact that due to line ending conversion, the actual contents of text files change, if the file is uploaded in text mode.
Pick the source folder carefully! You will not want to backup huge amount of data to a remote server, this can take a long time. Rather, gather your most important data to a single folder and do a remote backup on just that.

This poses a problem for the first time sync through FTP. We recommend that if possible, you start with an empty destination directory and let AASync transfer everything for the first time. This is the only way for AASync to know that its local sync database reflects reality on the remote side. Otherwise, there is a chance that the remote side may not be an exact copy of the source.

If sftp is used for syncing or it is a local-to-local sync this problem does not exist, you can start with a destination directory in any state, the sync should work.

Symbolic Link handling:

Symbolic Links are special kind of animals. They are not files, simply a pointers to a files or folders. The sync operation creates links identical to the ones in the source folder, the destination of the link does not get copied! In one case, Local-to-Remote sync through FTP, this is not possible because FTP does not include the capability to create links. In this case links are igored.

Case sensitivity Be sure to understand whether the source and the destination folders are on case sensitive file systems. If your source is on UFS and have files like foo.html and Foo.html, and your destination is an ntfs (not case sensitive) file system, these files WILL overwrite each other.