Technical Article
Writer: Laren Crawford
Published: November 2009
Applies to: Microsoft Sync Framework 2.0
Summary: This article describes how to synchronize efficiently with a remote replica by using a proxy provider on the local computer. The proxy provider uses the Remote Change Application pattern and Windows Communication Foundation (WCF) to send serialized metadata and data to the remote replica so synchronization processing can be performed on the remote computer with fewer round trips between the client and server computers.
Introduction
Microsoft Sync Framework synchronizes data between data stores. Typically, these data stores are on different computers or devices that are connected over a network. The application that controls synchronization can be on a third computer or can be on the same computer as one of the data stores. When you implement your own custom provider to represent a remote data store, there are two ways to perform synchronization:
· The remote computer does not run Sync Framework. The provider that represents the remote data store runs on the computer local to the synchronization application. The provider either applies changes individually to the remote data store, resulting in lots of round trips, or implements a custom batching mechanism.
· The remote computer runs Sync Framework and the local computer uses a proxy provider. The proxy provider performs minimal processing and uses the Remote Change Application pattern to send metadata and data to the provider on the remote computer, where change application is performed. This pattern allows the bulk of synchronization processing to be distributed to the remote computer. By caching change data, only one round trip is needed to process each change batch.
To demonstrate how to implement a proxy provider that uses the Remote Change Application pattern, this article uses WCF to send data and commands over the Internet to a remote provider running on a Web server. The C# code examples used in this article are taken from the Sync101 with Remote Change Application over WCF sample on the MSDN Code Gallery. To download the full sample, click here: Sync101 with Remote Change Application over WCF.
For more information about custom providers, see Custom Provider Fundamentals in the Sync Framework documentation.
Security Note: Sync Framework does not provide authentication or encryption between the proxy provider and the remote provider. To help prevent unauthorized access or tampering, the communication channel between the proxy provider and the remote provider must be secured by using an appropriate mutual authentication and encryption mechanism, such as Secure Sockets Layer (SSL).
Architecture of the Sample
The following figure shows a synchronization session that is running on the same computer as one of the replicas to synchronize. Using WCF, the proxy provider sends and receives commands and data to the remote provider that runs on the remote computer. The computer boundary is represented by a dashed red line.
Figure 1: Synchronizing by Using a Proxy Provider and Remote Provider
WCF requires several components to enable communication between the proxy provider and the remote provider. The following table lists the WCF components and their corresponding components in the code examples.
WCF Component / Code Example ComponentService contract interface / ISync101WebService
Service contract implementation on the Web server / Sync101WebService
Service contract client on the local computer / Sync101WebServiceClient
Defining the WCF Service Contract
Before the proxy provider can make calls to the remote provider, you must define a WCF service contract. This example defines the ISync101WebService interface, which contains methods that are similar to the Microsoft.Synchronization.KnowledgeSyncProvider methods, with appropriate adaptations to use serialized data where necessary. The similarity to KnowledgeSyncProvider methods is a convenience only: your interface can use whatever methods are most appropriate for your Web service. A partial listing of the service contract interface follows:
[ServiceContract(
Name = "Sync101WebService",
Namespace = "http://microsoft.synchronization",
SessionMode = SessionMode.Required)]
[ServiceKnownType(typeof(SyncIdFormatGroup))]
[ServiceKnownType(typeof(SyncId))]
public interface ISync101WebService
{
[OperationContract(
IsInitiating = true,
IsTerminating = false)]
void CreateProviderForSyncSession(
string dataAndMetadataFolderPath,
string storeName);
[OperationContract(
IsInitiating = false,
IsTerminating = false)]
ChangeBatch GetChangeBatch(
uint batchSize,
SyncKnowledge destinationKnowledge,
out Sync101.CachedChangeDataRetriever changeDataRetriever);
[OperationContract(
IsInitiating = false,
IsTerminating = false)]
byte[] ProcessChangeBatch(
ConflictResolutionPolicy resolutionPolicy,
ChangeBatch sourceChanges,
Sync101.CachedChangeDataRetriever changeDataRetriever,
byte[] changeApplierInfo);
}
These methods differ from their KnowledgeSyncProvider counterparts in several important ways:
Method / Differences from KnowledgeSyncProviderGetChangeBatch / Uses a custom Sync101.CachedChangeDataRetriever object. This object caches the data for all non-delete changes so that the change data is included with the change batch. This allows synchronization for a change batch to be performed with a single round trip between the local and remote computer, rather than requiring retrieval of item data for each item as it is applied.
ProcessChangeBatch / Uses a Sync101.CachedChangeDataRetriever object, and uses serialized change applier data in place of a NotifyingChangeApplier object. Returns serialized change applier data so the proxy provider can return it to Sync Framework. Also, the SyncCallbacks and SyncSessionStatistics objects are not passed because implementing callbacks would result in additional trips between the client and the server, and the session statistics are returned to the client as part of the serialized change applier data.
CreateProviderForSyncSession / This is a new method, not part of KnowledgeSyncProvider. This method initializes the Web service for synchronization.
Implementing the Service Contract Interface
The Sync101WebService class implements the ISync101WebService interface. The Web service uses a Sync101WebService object to handle calls from the client. This class is declared as follows:
[ServiceBehavior(
ConcurrencyMode = ConcurrencyMode.Single,
InstanceContextMode = InstanceContextMode.PerSession,
IncludeExceptionDetailInFaults = true)]
public class Sync101WebService : ISync101WebService
Most of the methods in the Sync101WebService class do nothing more than pass the call to an underlying KnowledgeSyncProvider implementation. The remaining methods are described in the following sections.
Initializing the Provider
The ISync101WebService service contract defines a method, CreateProviderForSyncSession, that initializes the Web service for synchronization. This method creates a MySyncProvider object (which implements KnowledgeSyncProvider) and initializes it with the path to the replica to be synchronized:
public void CreateProviderForSyncSession(string dataAndMetadataFolderPath, string name)
{
this.provider = new MySyncProvider(dataAndMetadataFolderPath, name);
}
Getting a Change Batch from the Remote Source Provider
When the remote provider is the source provider, the GetChangeBatch or GetFullEnumerationChangeBatch method is called. The Sync101WebService.GetChangeBatch implementation of this method retrieves a batch of changes from the underlying MySyncProvider object, creates a CachedChangeDataRetriever object to contain the data for non-delete changes, and returns both the ChangeBatch and the CachedChangeDataRetriever objects. The GetChangeBatch method is shown here. The GetFullEnumerationChangeBatch method is implemented in a similar way.
public ChangeBatch GetChangeBatch(
uint batchSize,
SyncKnowledge destinationKnowledge,
out CachedChangeDataRetriever changeDataRetriever)
{
object dataRetriever;
ChangeBatch changeBatch = provider.GetChangeBatch(batchSize, destinationKnowledge, out dataRetriever);
changeDataRetriever = new CachedChangeDataRetriever(dataRetriever as IChangeDataRetriever, changeBatch);
return changeBatch;
}
The CachedChangeDataRetriever class implements the IChangeDataRetriever interface, which is used by the destination provider to apply changes. The CachedChangeDataRetriever constructor enumerates the changes in the change batch and caches the item data for each non-delete change in a System.Collections.Generic.Dictionary object. When the destination provider calls LoadChangeData to retrieve item data, the data is retrieved from the local System.Collections.Generic.Dictionary cache instead of retrieving it by making a WCF request to retrieve the data from the source provider.
The CachedChangeDataRetriever class can be passed by WCF because it is made serializable by using the Serializable attribute.
The full CachedChangeDataRetriever class follows:
[Serializable()]
public class CachedChangeDataRetriever : IChangeDataRetriever
{
private SyncIdFormatGroup idFormats;
private Dictionary<SyncId, ItemData> cachedData;
public CachedChangeDataRetriever(
IChangeDataRetriever changeDataRetriever,
ChangeBatchBase sourceChanges)
{
this.idFormats = changeDataRetriever.IdFormats;
this.cachedData = new Dictionary<SyncId, Sync101.ItemData>();
// Look at each change in the source batch.
foreach (ItemChange itemChange in sourceChanges)
{
if (itemChange.ChangeKind != ChangeKind.Deleted)
{
// This is not delete, so there is some data associated
// with this change.
// Create a UserLoadChangeContext to retrieve this data.
UserLoadChangeContext loadChangeContext = new UserLoadChangeContext(
idFormats,
itemChange);
// Retrieve the data (we know that our provider uses data of type ItemData).
ItemData itemData = changeDataRetriever.LoadChangeData(
loadChangeContext) as ItemData;
// Cache the data
cachedData.Add(itemChange.ItemId, itemData);
}
}
}
public SyncIdFormatGroup IdFormats
{
get
{
return this.idFormats;
}
}
public object LoadChangeData(LoadChangeContext loadChangeContext)
{
return cachedData[loadChangeContext.ItemChange.ItemId];
}
}
Applying Changes by Using a Remote Destination Provider
When the remote provider is the destination provider, either the ProcessChangeBatch or ProcessFullEnumerationChangeBatch method is called to process the change batch. The Sync101WebService.ProcessChangeBatch implementation of this method retrieves local versions of source items from the metadata store, deserializes the change applier data into a NotifyingChangeApplier object, and uses the NotifyingChangeApplier object to apply changes to the destination replica. During change application, the CachedChangeDataRetriever object is used by the change applier to retrieve item data for the destination provider. After change application is complete, the change applier data is serialized and returned back to the proxy provider. The ProcessChangeBatch method is shown here. The ProcessFullEnumerationChangeBatch method is implemented in a similar way.
public byte[] ProcessChangeBatch(
ConflictResolutionPolicy resolutionPolicy,
ChangeBatch sourceChanges,
CachedChangeDataRetriever changeDataRetriever,
byte[] changeApplierInfo)
{
return provider.ProcessRemoteChangeBatch(resolutionPolicy, sourceChanges, changeDataRetriever, changeApplierInfo);
}
The MySyncProvider class implements a custom ProcessRemoteChangeBatch method that handles serialization and deserialization of the change applier data. This method is implemented as follows:
public byte[] ProcessRemoteChangeBatch(
ConflictResolutionPolicy resolutionPolicy,
ChangeBatch sourceChanges,
CachedChangeDataRetriever changeDataRetriever,
byte[] changeApplierInfo)
{
_metadataStore.BeginTransaction();
// Get the local change versions from the metadata store.
IEnumerableItemChange> localChanges = _metadata.GetLocalVersions(sourceChanges);
NotifyingChangeApplier changeApplier = new NotifyingChangeApplier(_idFormats);
// The following step is required for remote change application.
changeApplier.LoadChangeApplierInfo(changeApplierInfo);
changeApplier.ApplyChanges(
resolutionPolicy,
sourceChanges,
changeDataRetriever,
localChanges,
_metadata.GetKnowledge(),
_metadata.GetForgottenKnowledge(),
this,
null, // Note that a sync session context is not passed.
new SyncCallbacks());
_metadataStore.CommitTransaction();
// Return the serialized change applier data.
return changeApplier.GetChangeApplierInfo();
}
Implementing a Client that Calls the Web Service
To communicate with the Web service, implement a class that inherits from System.ServiceModel.ClientBase. This class simplifies the task of communicating with the Web service. This example defines the Sync101WebServiceClient class, which is a thin client implementation of the ISync101WebService service contract interface. The Sync101WebServiceClient class is declared as follows:
public class Sync101WebServiceClient : System.ServiceModel.ClientBase<ISync101WebService>, ISync101WebService
The methods of Sync101WebServiceClient do nothing more than forward the calls to the underlying ClientBase object, which does the work of sending the WCF request. The BeginSession method is shown here as an example:
public void BeginSession()
{
base.Channel.BeginSession();
}
Sync Framework requires an object that implements KnowledgeSyncProvider to represent the remote replica. This example defines the RemoteProviderProxy class, which implements the KnowledgeSyncProvider methods to perform minimal processing, such as serialization, and call the complementary Sync101WebServiceClient methods. Sync Framework treats the RemoteProviderProxy object as a standard provider.
A partial listing of the RemoteProviderProxy class follows:
public class RemoteProviderProxy : KnowledgeSyncProvider
{
private Sync101WebServiceClient client;
public override ChangeBatch GetChangeBatch(
uint batchSize,
SyncKnowledge destinationKnowledge,
out object changeDataRetriever)
{
CachedChangeDataRetriever cachedChangeDataRetriever;
ChangeBatch changeBatch = this.client.GetChangeBatch(
batchSize,
destinationKnowledge,
out cachedChangeDataRetriever);
changeDataRetriever = cachedChangeDataRetriever;
return changeBatch;
}
public override void ProcessChangeBatch(
ConflictResolutionPolicy resolutionPolicy,
ChangeBatch sourceChanges,
object changeDataRetriever,
SyncCallbacks syncCallback,
SyncSessionStatistics sessionStatistics)
{
CachedChangeDataRetriever cachedChangeDataRetriever = new CachedChangeDataRetriever(
changeDataRetriever as IChangeDataRetriever,
sourceChanges);
byte[] newChangeApplierInfo = this.client.ProcessChangeBatch(
resolutionPolicy,
sourceChanges,
cachedChangeDataRetriever,
this.syncSessionContext.ChangeApplierInfo);
this.syncSessionContext.ChangeApplierInfo = newChangeApplierInfo;
}
}
The GetChangeBatch method is called when the remote replica is the source replica. The RemoteProviderProxy.GetChangeBatch implementation of this method first creates an empty CachedChangeDataRetriever object to receive the serialized item data for items in the change batch. It then calls the Sync101WebServiceClient.GetChangeBatch method. This method calls through the Web service to get the change batch and change data from the remote provider.
The ProcessChangeBatch method is called when the remote replica is the destination replica. The RemoteProviderProxy.ProcessChangeBatch implementation of this method creates a CachedChangeDataRetriever object and fills it with item data for the change batch from the source provider. The cached item data, the change batch, and the serialized change applier data are all sent to the Sync101WebServiceClient.ProcessChangeBatch method, which sends them through the Web service to be processed by the remote provider. The change applier data that is returned from the remote provider is returned to Sync Framework by setting the ChangeApplierInfo property of the SyncSessionContext object that Sync Framework passed to the RemoteProviderProxy.BeginSession method.
Putting It All Together
Because the proxy provider implements KnowledgeSyncProvider, Sync Framework can treat it exactly like any standard custom provider. The synchronization application sets the proxy provider as one of the providers to be synchronized by the SyncOrchestrator object and calls SyncOrchestrator.Synchronize:
static void DoBidirectionalSync(string nameA, string nameB)
{
SyncOperationStatistics stats;
KnowledgeSyncProvider providerNameA = GetProviderForSynchronization(nameA);
KnowledgeSyncProvider providerNameB = GetProviderForSynchronization(nameB);
// Set the provider's conflict resolution policy.
// Because we are doing remote sync, we don't want to see callbacks.
providerNameA.Configuration.ConflictResolutionPolicy = ConflictResolutionPolicy.DestinationWins;
providerNameB.Configuration.ConflictResolutionPolicy = ConflictResolutionPolicy.DestinationWins;
// Set the sync providers
SyncOrchestrator agent = new SyncOrchestrator();
agent.Direction = SyncDirectionOrder.DownloadAndUpload;
agent.LocalProvider = providerNameA;
agent.RemoteProvider = providerNameB;
stats = agent.Synchronize();
}
static KnowledgeSyncProvider GetProviderForSynchronization(string name)
{
// Return the real provider for endpoint A.
// Return the proxy provider for endpoints B and C.
if (name == providerNameA)
{
return new MySyncProvider(folderPathForDataAndMetadata, providerNameA);
}
else if (name == providerNameB || name == providerNameC)
{
return new RemoteProviderProxy(folderPathForDataAndMetadata, name, endpointConfigurationName);
}
else
{
throw new ArgumentOutOfRangeException("name");
}
}
Execution Flow
There are a lot of pieces at work now, so let’s take a second look at the flow of execution of a method call:
1. / Sync Framework calls a method on the RemoteProviderProxy object, such as GetChangeBatch.2. / The RemoteProviderProxy object performs minimal processing and calls the corresponding method in the Sync101WebServiceClient object.
3. / The Sync101WebServiceClient object uses its underlying System.ServiceModel.ClientBase object to send the method call over WCF to the Web service.
4. / The Sync101WebService object on the Web server computer receives the call through WCF and calls the method on the MySimpleProvider object on the Web server computer.
5. / The MySimpleProvider object performs the appropriate synchronization action and returns the requested data to the Sync101WebService object.
6. / The Sync101WebService object returns the data through WCF to the client computer.
7. / The ClientBase object on the client computer receives the data from WCF and returns it to the Sync101WebServiceClient object.
8. / The Sync101WebServiceClient object returns the data to the RemoteProviderProxy object.
9. / The RemoteProviderProxy object returns the data to Sync Framework.
Conclusion