© Copyright Robert Vasvari, 1993-2008.

Folder Syncronization

• Website maintenance - Normally you have on your desktop a copy of your website that you edit. Whenever you think you done with your changes for now, you want to publish to your website. This is extremely easy with AASync, the sync engine will automatically find your changes and publish your new and changed files only to the remote server. You may sync as often as you like.
• Backups - Make copies of your important data to remote servers or to others disks on the desktop system. Either way it is the same easy-to-use interface!

Folder Sync means a very simple thing: You have a source folder whose contents you edit. Whenever the sync operation is performed, the target folder will get the deltas only, so changed files are copied to the target folder. This sounds very simple, but first we have to clear the air about the different kinds of Folder Syncronizations that are possible. Here is a quick explanation of terms:

Source Folder: The folder whose contents you change (like your local copy of your website). When you are done changing it, you run a Local-to-Remote sync to publish your changes.

Destination(Target) Folder: The folder that you do not touch directly, the sync engine will publish your changes from the Source Folder into the Destination Folder. This target folder can reside on your local machine or on a remote host on which you have an FTP or SSH/SFTP account.

Phase 1, Sync by Source Database: In this phase, the target folder is not looked at all, the source folder's state is compared to the source folder's previous state. The new state is then stored in a Database file (this file is located in the sync's home directory HOME/Library/Application Support/AASync2/SYNC_NAME). This method has various advantages: the speed first of all, since there is no need for network access. Secondly the accuracy: we do not have to put up with the differences between various systems, comparing file names checksums, mod dates are relatively messy across the internet. The source Database comparison is very accurate, it can easily compare checksums and resource fork data too. The remote target folder is then accessed just to upload the changed files. The question is, is this enough? In most cases, yes, if we are reasonably certain that the target folder is there, it is more less in sync, and it is not being altered by somebody else! Also, this methods is not available for RTL sync for obvious reasons.

Phase 2, Sync by Source-Destination Comparison: In this phase, the target folder is accessed by some protocol, like FTP/SFTP/SSH, then source and target files are actually compared by the most accurate means possible. FTP is a poor protocol, but SSH actually allows a very accurate comparison, since checksums of the remote files are available on most systems. This phase is more time-consuming then Phase 1, but, when it completes, one can be reasonably sure that source and target are in sync. If you use SFTP, AASync will automatically detect if SSH shell access is available, if so, it will be used to get more accurate folder listings, checksums, etc.

More on Sync Options...

In this case both source and target folders reside on the local workstation. Either one can be on a mounted external drive, the point is that there is NO network protocol (like FTP) access involved in reaching either source or target folder. Many other sync apps call the target folder "remote folder" mistakenly even though it actually just another locally mounted folder.

In this case the target folder resides on a remote host. There is no need to have any custom software run on the remote host, all you need is a simple FTP or SSH/SFTP account. When you specify the target folder, the URL will contain the remote host, protocol (FTP or SFTP), username and full path of the target folder.

In this case the source folder resides on a remote host. This kind of sync operation requires SSH/SFTP account. FTP cannot support Remote-to-Local(RTL) sync, because the protocol is too poor to provide the necessary information. When you specify the source folder, the URL will contain the remote host, protocol (SFTP only), username and full path of the source folder.

How does it work?

Here is the Sync Panel:

This table shows all existing Sync Definitions. The name of the sync along with its status and last sync time is shown. The green check icon means the last sync has concluded without error. If there is an error or the sync has been aborted by the user, a Yellow Caution icon is shown. If the sync is run on a schedule, an alarm clock is shown after the name, if the sync is chained and start after another sync, the chain icon is shown.

Set up sync definitions by clicking the New or Edit Selected Sync buttons. That brings up the Options Panel. The bottom table shows all sync pairs that have been defined already. The Mode column shows the abbreviation of the sync mode (Auto, P1, P2, or 'M' for Mirror), the Type column shows the type of sync operation: L->L for Local-to-Local, L->R for Local-To-Remote and R->L for Remote-To-Local.

Things to know about Remote-To-Local Folder Sync

There are a number of limitations to RTL sync: FTP as a protocol is simply too limited to support RTL sync. This is due to the fact that most FTP servers reset modification dates when a file is transferred to it, checksum is not available and size alone simply cannot detect change to a file. Therefore, RTL sync is only available in using the SFTP protocol. RTL sync is a simple one phase operation, both remote source and local target are scanned once. The other limitation is that if a file is simply renamed in the source, RTL sync still has to treat it like a new file, so it is downloaded as if it were new. This is due to the fact that a rename cannot be detected deterministically just by reading remote data, since you can have more files with identical contents.

Things to know about Local-to-Remote Folder Sync

The sync engine in AASync employs a two-phase sync system for Local-to-Remote Folder Sync. In the first phase, it builds a database in a hidden file that is put in the source folder. This database records the state of the source and compares it to the previous database file. This way, changes can be detected easily without having to rely on various protocols to get information from the remote system. This is especially important if FTP is the protocol used to communicate with the remote system, since FTP is such a poor protocol, that it really does not allow to deterministically track changes. It trashes mod dates, does not have checksum and with text files even the file sizes change during uploads. Not to mention the fact that due to line ending conversion, the actual contents of text files change, if the file is uploaded in text mode.

This poses a problem for the first time sync through FTP. We recommend that if possible, you start with an empty target directory and let AASync transfer everything for the first time. This is the only way for AASync to know that its local sync database reflects reality on the remote side. Otherwise, there is a chance that the remote side may not be an exact copy of the source.

If sftp is used for syncing or it is a local-to-local sync this problem does not exist, you can start with a target directory in any state, the sync should work.

Symbolic Links are special kind of animals. They are not files, simply a pointers to a files or folders. The sync operation creates links identical to the ones in the source folder, the target of the link does not get copied! In one case, Local-to-Remote sync through FTP, this is not possible because FTP does not include the capability to create links. In this case links are ignored.

Case sensitivity Be sure to understand whether the source and the target folders are on case sensitive file systems. If your source is on UFS and have files like foo.html and Foo.html, and your target is an hfs file system, these files WILL overwrite each other.

Here is a summary of Folder Sync features:

• Local-to-Local(LTL), Local-To-Remote(LTR) and Remote-To-Local(RTL) Folder Sync
• Mirror or Archive mode
• Scheduled Backups with machine wake/sleep options
• Chaining syncs into a sequence
• Fork Data is preserved if possible
• Any protocol (SFTP or FTP) can be used
• Changed filenames are detected so unnecessary file transfers can be avoided.
• Easy drag-and_drop definitions of sync pairs.
• Any number of simultaneous sync operations!