Implementing WebDAV Synchronization

IT Hit WebDAV Server supports synchronization based on Sync-ID. To implement Sync-ID algorithm your server must be able to return all changes that happened in your storage since provided sync-token, including deleted items. In addition to that each item must have a unique ID and parent ID.

To test synchronization you can use samples samples provided with IT Hit User File System:

To support synchronization your storage must support the following functionality:

Each item must have ID and ParentID unique withing your file system.
Your storage must be able to store information about deleted items.
Your storage must be able to return all changes that happened since provided sync-token. Including it must return deleted items and distinguish between deleted and all other changes (created, updated and moved items).
Your server must provide access to items based on ID. For example via https://serv/ID/{1234567890} URL. This is required to sync with macOS and iOS devices.

To support synchronization the library provides ISynchronizationCollection, IBind and IChangedItem interfaces. You will implement IBind and IChangedItem on all items and ISynchronizationCollection on folder items that support synchronization of its content. Typically you will implement ISynchronizationCollection on your root folder and this will enable synchronization on your entire hierarchy.

The IBind interface provides item Id and ParentId properties. The IChangedItem interface provides a single ChangeType property. In the example below, for demo purposes, we will mark deleted items as hidden, so we can find them later, when changes are requested:

public abstract class ChangedItem: DavHierarchyItem, IBind, IChangedItem
{
    ...
    public Change ChangeType
    {
        get => fileSystemInfo.Attributes.HasFlag(FileAttributes.Hidden) ?
            Change.Deleted : Change.Changed;
    }    

    public string Id { get => $"{fsInfo.GetId()}"; }

    public string ParentId { get => $"{Directory.GetParent(fsInfo.FullName).GetId()}"; }
}

The ISynchronizationCollection interface provides GetChangesAsync() method that returns all changes below this folder since provided sync-token. Each item in the list of changes retuned by this method must implement IBind and IChangedItem interfaces. Below is the example of this method implementation based on file USN. USN changes across entire file system during every item change, providing a simple way to implement sync-token in case of file system back-end.

public class DavFolder : DavHierarchyItem, IFolder, ISynchronizationCollection
{
    ...
    public async Task<DavChanges> GetChangesAsync(
        IList<PropertyName> propNames, 
        string syncToken, 
        bool deep, long? 
        limit = null)
    {
        // In this sample we use item's USN as a sync token.
        // USN increases on every item update, move, creation and deletion. 

        DavChanges changes = new DavChanges();
        long syncId = string.IsNullOrEmpty(syncToken) ? 0 : long.Parse(syncToken);


        // Get all file system entries with usn.
        var childrenList = new List<Tuple<IChangedItem, long>>();
        foreach ((string Path, long SyncId) item in await GetSyncIdsAsync(syncId, deep))
        {
            // Get URI from file system path.
            string childUri = context.ReverseMapPath(item.Path);
                
            IChangedItem child = await DavHierarchyItem.GetItemAsync(context, childUri);

            if (child != null)
            {
                childrenList.Add(new Tuple<IChangedItem, long>(child, item.SyncId));
            }
        }

        // If limit==0 this is a sync-token request, no need to return any changes.
        bool isSyncTokenRequest = limit.HasValue && limit.Value == 0;
        if(isSyncTokenRequest)
        {
            changes.NewSyncToken = childrenList.Max(p => p.Item2).ToString();
            return changes;
        }

        IEnumerable<Tuple<IChangedItem, long>> children = childrenList;

        // If syncId == 0 this is a full sync request.
        // We do not want to return deleted items in this case.
        if (syncId == 0)
        {
            children = children.Where(item => item.Item1.ChangeType != Change.Deleted);
        }

        // Return new sync token.
        changes.NewSyncToken = children.Max(p => p.Item2).ToString();

        // Return changes.
        changes.AddRange(children.Select(p => p.Item1));

        return changes;
    }

    private async Task<IEnumerable<(string Path, long SyncId)>> GetSyncIdsAsync(
        long minSyncId, 
        bool deep)
    {
        // First we must read max existing USN. However, in this sample,
        // for the sake of simplicity, we just read all changes under this folder.

        SearchOption options = SearchOption.AllDirectories;

        var list = new ConcurrentBag<(string Path, long Usn)>();

        ParallelOptions parallelOptions = new()
        {
            MaxDegreeOfParallelism = 1000
        };

        var decendants = Directory.GetFileSystemEntries(dirInfo.FullName, "*", options);
        await Parallel.ForEachAsync(decendants, parallelOptions, async (path, token) =>
        {
            long syncId = await new FileSystemItem(path).GetUsnByPathAsync();
            if (syncId > minSyncId)
            {
                list.Add(new(path, syncId));
            }
        });

        return list;
    }
}

In addition to the list of changes the GetChangesAsync() method returns a new sync-token. The client will store the sync token and use it to request changes during next request, for example when the server will notify the client that changes are available.

In case the syncToken parameter is null, this means the client is doing an initial synchronization. You must return all items under this folder. Note that you do not need to return deleted items in this case.

The limit parameter indicates maximum number of items that should be returned from this method. If you truncate the results, you must indicate that more changes are available by setting DavChanges.MoreResults to true.

If the limit is null, this means the client did not specify limit and you must return all items.

If the limit parameter is 0, this means the client needs the sync-token only. You should not return any items in this case, instead you will only set the DavChanges.NewSyncToken property and the Engine will return it to the client. This scenario is typical for clients with on-demand population support. In case of the on-demand population, during start, the client must first read the sync-token from server and than it can start populating some of its folders.

The deep parameter indicates if this folder children should be returned or entire hierarchy under this folder.

.NET Server

Implementing WebDAV Synchronization

In this article

Implementing WebDAV Synchronization

See Also:

Next Article: