Synchronization

The main goal of SynqClient is to provide means to synchronize a local and a remote folder. Such a folder may contain arbitrary files and also sub-folders. However, it is assumed that the structure to be synchronized is well-defined and does not change too much over time. In particular, the sync procedure does not cover the following:

Moves and copies of files or entire folders are not detected. Hence, if they occur, the library will simply upload/download the duplicate/moved data.
Changing a path from being a file to a folder or vice versa is not supported.

This is basically by design: The use case for the synchronization as implemented in SynqClient is to allow a program to store its state in a well defined folder structure and keep it in sync between devices. Such a programatically created structure usually has a quite fixed design. For example, the OpenTodoList app, form which this library has been factored out, used the following approach:

The app organized various items - notes, images and todo lists - in libraries.
Each such library is represented as a folder consisting of sub-folders and files.
When a new item is created, it gets a path assigned using the following approach: <library_root_folder>/<year>/<month>/<item_file_name>.txt. - In other words, inside the library folder, we have in the first level below one folder per year in which an item has been created. Inside each year’s folder we have one folder per month. Finally, inside the per-month folder, we put the actual files.

As you can see, this plays well with the limitations of the sync procedure: Items never get moved around. Additionally, there will never be a case where a path is converted from a file to a folder or vice versa.

If you can design your app to store its information in a similar manner, than SynqClient should work just fine for you to keep the app state in sync between devices!

The rest of this chapter will list all the ingredients we need to sync a local and a remote folder.

Remote Access

The most important ingredient is access to the “remote” folder. For this, the library provides a job based interface which should allow to implement access against a wide range of servers and protocols. The base class for all jobs is SynqClient::AbstractJob. From this, specific (but still abstract) job types like the SynqClient::GetFileInfoJob, SynqClient::DownloadFileJob or SynqClient::DeleteJob are derived. To implement access to a concrete server or talk over a specific protocol, these abstract classes need to be implemented. These concrete implementations can either be part of the SynqClient library or can be part of the host application (or a higher level library) which uses SynqClient.

Factories

The actual sync functionality is written in a high level manner, meaning: On that level, no details about the underlying protocol to access remote files is used (or desired). Consequentially, this part is written entirely against the abstract job classes mentioned above. However, as the sync functionality needs to create jobs on the fly, factories are used. Factories are subclasses of the SynqClient::AbstractJobFactory class.

AbstractJobFactory

The AbstractJobFactory class is the base class for concrete factories that are then used in conjunction with the code for the actual synchronization. It just defines the available protocol but on its own it does not provide any functionality.

class AbstractJobFactory : public QObject

Factory for jobs.

This class is the abstract base for concrete factories which produce jobs that can be used for a concrete protocol or server. On is own, this class does nothing (but provide a nice interface around the virtual part of the class).

Concrete sub-classes must implement the interface defined by this class in order to allow working with a specific protocol or backend server.

Subclassed by SynqClient::DropboxJobFactory, SynqClient::WebDAVJobFactory

Public Functions

explicit AbstractJobFactory(QObject *parent = nullptr): Constructor.

~AbstractJobFactory() override: Destructor.

CreateDirectoryJob *createDirectory(QObject *parent = nullptr)

Create a job to create a new folder.

This creates a job which creates a folder remotely. The resulting object will be owned by the parent. If creating the job fails, a nullptr is returned.

DeleteJob *deleteResource(QObject *parent = nullptr)

Create a job to delete a file or directory.

This creates a job which deletes resources remotely. The resulting object will be owned by the parent. If creating the job fails, a nullptr is returned.

DownloadFileJob *downloadFile(QObject *parent = nullptr)

Create a job to download a file.

This creates a job which downloads a file. The resulting object will be owned by the parent. If creating the job fails, a nullptr is returned.

UploadFileJob *uploadFile(QObject *parent = nullptr)

Create a job to upload a file.

This creates a job which upload a file. The resulting object will be owned by the parent. If creating the job fails, a nullptr is returned.

GetFileInfoJob *getFileInfo(QObject *parent = nullptr)

Create a job to fetch information about a single file or folder.

This creates a job which gets information about a single remote file or directory. The resulting object will be owned by the parent. If creating the job fails, a nullptr is returned.

ListFilesJob *listFiles(QObject *parent = nullptr)

Create a job to list remote files.

This creates a job which lists the entries in a remote directory. The resulting object will be owned by the parent. If creating the job fails, a nullptr is returned.

RemoteChangeDetectionMode remoteChangeDetectionMode() const

Returns the mode that is used to detect remote changes.

This returns the mode that is used for the backend represented by the factory to detect changes on the remote server.

See also

setRemoteChangeDetectionMode

bool alwaysCheckSubfolders() const

Does a sync always need to follow sub-folders?

This property indicates if a sync needs to always check all subfolders for detecting changes.

See also

setAlwaysCheckSubfolders

Protected Functions

explicit AbstractJobFactory(AbstractJobFactoryPrivate *d, QObject *parent = nullptr): Constructor.

virtual AbstractJob *createJob(JobType type, QObject *parent = nullptr) = 0

Create a new job.

Concrete factory classes must override this method to actually create job objects. The type of job to create is passed in, as well as the parent object which the resulting object shall belong to.

On success, the created job shall be returned; on error, a nullptr.

void setRemoteChangeDetectionMode(RemoteChangeDetectionMode mode)

Set the mode used to detect remote changes during synchronization.

This sets the mode that is used by synchronization code to detect changes on the remote server. Different backends behave differently, so the way remote changes are detected efficiently might require backend specific handling.

The default value is RemoteChangeDetectionMode::FoldersWithSyncAttributes, but concrete factories might call this in their constructor to change the mode something else if the backend they represent need specific handling.

Note that the default mode should in general work with any backend, however, a more efficient handling (usually resulting in fewer queries that need to be run) might be possible by changing to a different mode.

void setAlwaysCheckSubfolders(bool alwaysCheckSubfolders)

Set if a sync shall always check subfolders.

This method can be used to indicate to a synchronization mechanism such as DirectorySynchronizer, that subfolders always have to be checked. For example, when using WebDAV style sync, usually, sync attributes are assumed to change if there is a change either directly or recursively the sync attribute of a folder changes. If this flag is set to true, subfolders will always be followed and checked for changes, even if the server reports unchanged sync attributes.

The SynqClient::JobType enum is used by the protected class interface to determine which concrete kind of job shall be created:

enum class SynqClient::JobType : quint32

Used to identify a specific type of jobs.

Values:

enumerator Invalid: Indicates an invalid job.

enumerator CreateDirectory: A job to create a new directory.

enumerator DeleteResource: A job to delete a file or directory.

enumerator DownloadFile: A job to download a file.

enumerator UploadFile: A job to upload a file.

enumerator GetFileInfo: A job to get information about a single file or directory.

enumerator ListFiles: A job to get information about entries in a folder.

Concrete Factories

The following concrete factory classes are provided by SynqClient in order to allow synchronization against specific backend servers and protocols:

class WebDAVJobFactory : public SynqClient::AbstractJobFactory 

Create jobs to talk to a WebDAV server.

This factory is used to create jobs which can be used to talk with a concrete WebDAV server. The factory needs to be configured: At least a networkAccessManager() must be set as well as the base url() of the server to talk to.

Public Functions

explicit WebDAVJobFactory(QObject *parent = nullptr): Constructor.

~WebDAVJobFactory() override: Destructor.

QNetworkAccessManager *networkAccessManager() const: The network access manager used by jobs created by the factory.

void setNetworkAccessManager(QNetworkAccessManager *networkAccessManager): Set the network access manager to be used by created jobs.

QUrl url() const: The base URL of the server to connect to.

void setUrl(const QUrl &url): Set the base URL of the server to connect to.

QString userAgent() const: The default user agent used by jobs created by the factory.

void setUserAgent(const QString &userAgent): Set the default user agent for jobs created by the factory.

WebDAVServerType serverType() const: The type of WebDAV server to connect to.

void setServerType(WebDAVServerType serverType): Set the type of WebDAV server to connect to.

WebDAVWorkarounds workarounds() const

Workarounds required for using the server.

This property holds the required workarounds needed to work with the server. They can be manually set or automatically retrieved using testServer().

void setWorkarounds(WebDAVWorkarounds workarounds): Set the required workarounds for the server.

int transferTimeout() const

The transfer timeout before aborting requests.

This is the timeout before aborting network jobs when no transfer takes place.

See also

AbstractJob::transferTimeout()

void setTransferTimeout(int transferTimeout): Set the transfer timeout.

Public Slots

void testServer(const QString &path = QString())

Test the server.

This starts testing the server. WebDAV servers differ heavily in the way the implement the WebDAV standard; some might even be unsuitable for running the synchronization.

This function starts running some tests against the server, checking if it provides the necessary functions that are needed to run a successful sync.

Once the tests finish, the serverTestFinished() signal is emitted. The results of the tests are stored in the workarounds() property and can be saved and later on restored.

The optional path argument is the path on the server, where (temporary) files and folders will be created in. Note that the remote path must exist, otherwise, the tests will fail.

Protected Functions

explicit WebDAVJobFactory(WebDAVJobFactoryPrivate *d, QObject *parent = nullptr): Constructor.

virtual AbstractJob *createJob(JobType type, QObject *parent) override: Implementation of AbstractJobFactory::createJob().

class DropboxJobFactory : public SynqClient::AbstractJobFactory 

Public Functions

explicit DropboxJobFactory(QObject *parent = nullptr): Constructor.

~DropboxJobFactory() override: Destructor.

QNetworkAccessManager *networkAccessManager() const

The network access manager to be used by created jobs.

This property holds the QNetworkAccessManager that all created jobs of this factory will be using by default.

void setNetworkAccessManager(QNetworkAccessManager *networkAccessManager): Set the networkAccessManager to be used by jobs created by this factory.

QString userAgent() const: The user agent used by the created jobs.

void setUserAgent(const QString &userAgent): Set the userAgent to be used by the created jobs.

QString token() const: The OAuth2 bearer token to be used to authenticate against the Dropbox API.

void setToken(const QString &token): Set the OAuth2 bearer token to be used to authenticate against the Dropbox API.

int transferTimeout() const

The transfer timeout before aborting network requests.

This is the timeout if no network transfer occurs after which a network request will be aborted.

See also

AbstractJob::transferTimeout()

void setTransferTimeout(int transferTimeout): Set the transfer timeout.

Protected Functions

explicit DropboxJobFactory(DropboxJobFactoryPrivate *d, QObject *parent = nullptr): Constructor.

virtual AbstractJob *createJob(JobType type, QObject *parent) override: Implementation of SynqClient::AbstractJobFactory::createJob().

Synchronization State Database

The next very important ingredient for the sync is the State Database. In order to implement proper synchronization, we need to keep track of some information between sync runs. This information must be saved persistently. In order to be as flexible as possible, the functionality to interact with such a database is done via another interface: The SynqClient::SyncStateDatabase class.

SyncStateDatabase

class SyncStateDatabase : public QObject

Store synchronization state information between sync runs.

This class is an abstract base for classes which serve as databases for persistently storing synchronization state information. In order to identify changes and decide if files need to be updated, some state information must be stored between sync runs. This class serves as interface for such classes, so applications can opt to use any type of storage for such information.

Subclassed by SynqClient::JSONSyncStateDatabase, SynqClient::SQLSyncStateDatabase

Public Functions

explicit SyncStateDatabase(QObject *parent = nullptr): Constructor.

~SyncStateDatabase() override: Destructor.

virtual bool openDatabase()

Open the database for a sync run.

This method is called once to indicate that a sync starts. On success, it shall return true. When an error occurs, it shall return false.

The default implementation does nothing. Concrete sub-classes may override this method to acquire resources or initialize the database.

See also

closeDatabase()

virtual bool addEntry(const SyncStateEntry &entry) = 0

Write the entry to the database.

This adds the given entry to the database (or overrides it, if it already is present). Returns true on success or false otherwise.

virtual SyncStateEntry getEntry(const QString &path) = 0

Get a single entry from the database.

This method returns the entry having the given path. If no such entry is stored in the database, an invalid entry is returned.

virtual QVector<SyncStateEntry> findEntries(const QString &parent, bool *ok = nullptr) = 0

Get all direct children of the entry with the given path.

This returns a list of state entries which are direct children of the entry identified by the path. This only includes direct children.

If ok is set to a valid pointer, this writes true to the pointed to variable to indicate success or false otherwise.

virtual bool removeEntries(const QString &path) = 0

Recursively delete entries from the database.

This recursively removes all items which are children of the item identified by the path.

Returns true on success or false of there was any error.

virtual bool removeEntry(const QString &path) = 0

Remove a single entry from the database.

This removes the entry identified by the path from the database. On success, this returns true. On error, false is returned.

Note

If the path refers to a directory, this will only remove the information about the directory itself. Data about children will still be kept in the database.

virtual bool closeDatabase()

Close the database.

This is the counterpart to openDatabase(). It is called after a sync. Sub-classes may use it to write out data, release resources and so on. On success, it shall return true, on error, false.

The default implementation does nothing.

See also

openDatabase()

bool isOpen() const

Get if the database open.

This returns true if the database previously has been opened successfully via openDatabase().

bool iterate(std::function<void(const SyncStateEntry &entry)> callback, const QString &path = "/")

Iterate over the entries in the database.

This is a utility method which calls the callback for all nodes found in the database starting at the given root path.

The function returns true on success or false if - during iteration - an error occurred.

Protected Functions

explicit SyncStateDatabase(SyncStateDatabasePrivate *d, QObject *parent = nullptr): Constructor.

The SynqClient::SyncStateEntry class is used to store information about a single entry in the sync database:

class SyncStateEntry

Holds state information for a single file or folder.

This class holds the path of a file or folder as well as additional local and remote status information, which is used in each sync run to decide if the item needs to be updated.

Public Functions

SyncStateEntry(): Constructor.

SyncStateEntry(const SyncStateEntry &other): Copy constructor.

SyncStateEntry(const QString &path, const QDateTime &modificationTime, const QString &syncProperty)

Constructor.

Creates a new entry with the given path, modificationTime and syncProperty. The entry will be valid (i.e. isValid() returns true).

virtual ~SyncStateEntry(): Destructor.

SyncStateEntry &operator=(const SyncStateEntry &other): Assignment operator.

bool isValid() const

Used to indicate if the entry is valid.

This property can be used to indicate if the entry is valid. By default, an entry is initialized with a value of false. APIs that look up entries and return them shall set it to true on returned entries.

void setValid(bool valid): Set the valid property.

QString path() const

The file path.

This is the path of the entry in a synchronized directory. The path is of the form /parent/sub/othersub/file.txt, i.e.:

It is stored absolute (and implicitly assumed to be relative to the root of the local/remote directory to be synced).
It uses forward slashes on all platforms.
The top level folder is represented as /.
For all folders except the top level one, the path must not end with a slash.

If a path is not in that form, it will be converted accordingly by this class upon setting the path.

void setPath(const QString &path): Set the path of the file or folder.

QDateTime modificationTime() const

Holds the last modification date and time.

This property holds the date and time of the last modification that was done locally. This is required to detect changes to the file that happen in the local directory.

void setModificationTime(const QDateTime &modificationTime): Set the date and time of the last local modification.

QString syncProperty() const

The remote sync property.

This property holds the value of the remote sync property (converted to a string). The sync property must be reported by jobs retrieving the remote file information. It is supposed to be a property which can be used to detect remote changes. The concrete value and form depends on the backend used. For example, for WebDAV, this property could be the etags reported by the server for the files and folders.

void setSyncProperty(const QString &syncProperty): Set the remote sync property of the file or folder.

Public Static Functions

static QString makePath(const QString &path)

Convert a path to a sync entry path.

This is a helper function which returns a path suitable to be used with the sync state database API. Internally, this is used by setPath() to make sure paths set are stored in a suitable format internally.

This method can also be used by other parts to create suitable paths similar to the ones used internally by this class.

static QString makePath(const QDir &dir, const QString &path)

Convert path to a sync entry path.

This is an overloaded version of makePath. It takes a dir and a path (which is supposed to be relative to the directory) and converts it to a sync entry path.

Concrete Databases

Often, the functionality of such a synchronization database won’t be different between applications. For this reason, SynqClient comes with the following default implementations, which can be used out of the box instead of implementing own ones:

class SQLSyncStateDatabase : public SynqClient::SyncStateDatabase 

Store sync state information in an SQL database.

This class can be used to store the sync state information in an SQL database. This can either be a dedicated database (such as a single SQLite database file) or a database in an SQL server such as MySQL, PostgreSQL and so on.

Public Functions

explicit SQLSyncStateDatabase(QObject *parent = nullptr): Constructor.

explicit SQLSyncStateDatabase(const QSqlDatabase &db, QObject *parent = nullptr)

Constructor.

This is an overloaded version of the constructor. It accepts the database to be used to store state.

See also

setDatabase()

explicit SQLSyncStateDatabase(const QString &path, QObject *parent = nullptr)

Constructor.

This is an overloaded version of the constructor. It accepts the path to the SQLite database file to be usd to store data.

~SQLSyncStateDatabase() override: Destructor.

QSqlDatabase database() const: The SQL database used to store state.

void setDatabase(const QSqlDatabase &database)

Set the SQL database used to store state.

This sets the database to be used to write persistent sync information into. Please note, that only the connection name will be stored internally.

void setDatabase(const QString &path)

Set the path to the SQLite datbase file to be used to store state.

This is an overloaded setter. When being used, it will create a new SQLite database connection which uses the file path to store information.

virtual bool openDatabase() override

Open the database.

This opens the sync state database. If the underlying SQL database connection is not yet open, this will also open that connection.

Returns true on success or false on error.

virtual bool addEntry(const SyncStateEntry &entry) override: Implementation of SyncStateDatabase::addEntry().

virtual SyncStateEntry getEntry(const QString &path) override: Implementation of SyncStateDatabase::getEntry().

virtual QVector<SyncStateEntry> findEntries(const QString &parent, bool *ok) override: Implementation of SyncStateDatabase::findEntries().

virtual bool removeEntries(const QString &path) override: Implementation of SyncStateDatabase::removeEntries().

virtual bool removeEntry(const QString &path) override: Implementation of SyncStateDatabase::removeEntry().

virtual bool closeDatabase() override

Close the database.

This closes the sync state database. Internally, this will close the connection to the underlying SQL database.

Returns true on success or false otherwise.

Protected Functions

explicit SQLSyncStateDatabase(SQLSyncStateDatabasePrivate *d, QObject *parent = nullptr): Constructor.

class JSONSyncStateDatabase : public SynqClient::SyncStateDatabase 

Store persistent sync state information in a single JSON file.

This class can be used to store sync state information in a JSON file on disk. In order to be usable, a file must be set from which to read and into which to write to state information.

Public Functions

explicit JSONSyncStateDatabase(const QString &filename, QObject *parent = nullptr)

Constructor.

Creates a new JSON sync state database which saves its data to the given filename.

explicit JSONSyncStateDatabase(QObject *parent = nullptr)

Constructor.

Creates an empty JSON sync state database. Use setFilename() to set the path to the file where to save to and load from persistent information.

~JSONSyncStateDatabase() override: Destructor.

QString filename() const: The path to the file used to hold persistent information.

void setFilename(const QString &filename): Set the path to the file where to store persistent information.

virtual bool openDatabase() override

Open the sync state database.

This will try to open the JSON file referred to by the filename() property. If the file does not exist, it will be created and opened. If opening or creating the file fails, this method returns false.

virtual bool closeDatabase() override: Implementation of SyncStateDatabase::closeDatabase().

virtual bool addEntry(const SyncStateEntry &entry) override: Implementation of SyncStateDatabase::addEntry().

virtual SyncStateEntry getEntry(const QString &path) override: Implementation of SyncStateDatabase::getEntry().

virtual QVector<SyncStateEntry> findEntries(const QString &parent, bool *ok) override: Implementation of SyncStateDatabase::findEntries().

virtual bool removeEntries(const QString &path) override: Implementation of SyncStateDatabase::removeEntries().

virtual bool removeEntry(const QString &path) override: Implementation of SyncStateDatabase::removeEntry().

Protected Functions

explicit JSONSyncStateDatabase(JSONSyncStateDatabasePrivate *d, QObject *parent = nullptr): Constructor.

Directory Synchronizer

With the above mentioned ingredients, we have everything at hand required to implement synchronization of a local and a remote folder. The synchronization part is implemented as a dedicated class - SynqClient::DirectorySynchronizer - which is configured appropriately with concrete instances of the functional blocks described above.

class DirectorySynchronizer : public QObject

Synchronizes a local and a remote directory.

This class is used to keep two directories - a local one and a remote one - synchronized.

Synchronizer Life Cycle

The usage pattern of the synchronizer is actually quite similar to that of jobs:

After creating a new instance of this class, it is in the SynchronizerState::Ready state. In this state, it must be set up, i.e. at least the following attributes must be set:

jobFactory()
syncStateDatabase()
localDirectoryPath()
remoteDirectoryPath()

In addition, other properties can be changed to alter the bahviour of the synchronization.

Once all inputs are provided, the start() method can be called. This will transition the synchronizer to the SynchronizerState::Running state. The method returns immediately. Eventually, the finished() signal will be emitted to indicate that the sync is done and the synchronizer transitioned to the SynchronizerState::Finished state.

While the synchronizer is running, the stop() method can be used to stop the synchronization.

Error Handling

This is also similar to the job interface: The synchronizer provides the error() method, which returns an error ID indicating any issues occurred during the synchronization. Note that only the first error that occurred will be reported by that. This is important to keep in mind, as the synchronizer will spawn multiple jobs in parallel to improve overall throughput and make use of pipelining when using network jobs.

In addition, the errorString() method can be used to retrieve a textual representation of the error that occurred. This might also contain valuable information from underlying jobs that failed.

Public Types

typedef std::function<bool(const QString &path, const FileInfo &fileInfo)> Filter

Type definition for file filters.

This typedef defines the signature of callables suitable to be used as file filters. A filter function gets two parameters:

The path to the file or folder. This is usually formatted as an absolute path with forward slashes only. This path is interpreted to be relative to the local and/or remote folder which is synchronized.
The FileInfo record, which potentially holds more information about the file or folder. In particular, it holds the information if a path refers to a file or a folder.

The function shall return true if the file or folder shall be included in the sync.

Note

If for a folder the filter returns false, the sync will exclude all files and folders below that one recursively. Hence, if you need to skip e.g. files directly below a folder but include ones further down the same hierarchy, you must return true for the folder itself.

Public Functions

explicit DirectorySynchronizer(QObject *parent = nullptr): Constructor.

~DirectorySynchronizer() override: Destructor.

AbstractJobFactory *jobFactory() const

The factory used to created jobs to access remote files.

This returns the factory which is used to create the jobs necessary to access remote files and folders. By default, this is a nullptr. Before using the synchronizer, this must be set to a valid factory.

void setJobFactory(AbstractJobFactory *jobFactory): Set the factory used to create jobs for accessing remote files.

SyncStateDatabase *syncStateDatabase() const

The persistent sync state storage.

This holds the database used to persistently store sync state information. This is required to detect both local and remote changes between sync runs. By default, this is a nullptr and must be set to a valid database before using the synchronizer.

void setSyncStateDatabase(SyncStateDatabase *syncStateDatabase): Set the persistent sync state storage.

QString localDirectoryPath() const

The path to the local directory to sync.

This is the path to the local directory which shall be synchronized. By default, this is an empty string. To use the synchronizer, set it to a valid folder path.

Note

The local directory must exist.

void setLocalDirectoryPath(const QString &localDirectoryPath): Set the path to the local folder to be synchronized.

QString remoteDirectoryPath() const

The path to the remote directory to be synchronized.

This is the path to the remote path that shall be synchronized. Please note that the path specified might be relative to a root folder which is part of the configuration of the used jobFactory().

By default, this is empty. Set it to a valid string. If you need to synchronize against the root folder configured in the job factory, use /.

void setRemoteDirectoryPath(const QString &remoteDirectoryPath): Set the path to the remote directory to be synchronized.

Filter filter() const

A filter determining files and folders to be included in the synchronization.

This returns the filter that is used during the synchronization to determine which files and folders to include in the synchronization. The filter is a simple function which gets the path to the file or folder and a FileInfo object and returns either true if that file/folder shall be included in the synchronization or false otherwise. The paths passed to the filter are formatted as absolute paths with forward slashes. The path is implicitly relative to the configured local and remote directory.

The default filter returns true for every path passed into it.

If you plan to put the data files for the syncStateDatabase() in the local folder that is synchronized, you have to configure a suitable filter as well. Otherwise, the synchronization database is synced as well - which is almost certainly never what you intent.

For example, if you plan to use the JSONSyncStateDatabase and you want to store it as sync.json inside the folder that is synchronized, you can use the following to exclude the databse from sync:

DirectorySynchronizer sync;
sync.setFilter([](const QString &path, const SynqClient::FileInfo &fileInfo) {
    if (path == "/sync.json") {
        // Exclude the sync state database file from the sync:
        return false;
    }
    // Include every other file:
    return true;
});

See also

SyncStateEntry::makePath()

void setFilter(const Filter &filter): Set the filter used to determine which files and folders to include in the sync.

int maxJobs() const

The maximal number of jobs to spawn in parallel.

This is the maximal number of jobs that are created in parallel during the sync. By default, this is set to 12. This is in line with the QNetworkAccessManager API: Internally, it will establish at most 6 connections to the same host/port pair in parallel. As jobs might have some “dead” time during which no other network connection takes place, we use double that number of jobs to ensure that we utilize parallelization and pipelining in an optimal way to speed up network transfers.

When using a job factory which does not internally use QNetworkAccessManager, adjust this value accordingly. In particular, in case you have a job implementation which requires sequential access, you should set this value to 1.

void setMaxJobs(int maxJobs): Set the maximal number of jobs to spawn in parallel.

bool retryWithFewerJobs() const

Indicates that the sync should be retried with fewer parallel jobs.

If this flag is set after a sync, this indicate that the remote had issues and hence we could potentially retry the sync with fewer parallel workers (if possible). Users of the class should check if the sync finished with this error and retry it, with the maximum number of parallel jobs reduced (in the worst case to e.g. only 1 worker). This sometimes is necessary when the server used is not capable to handle too many requests in parallel and hence starts closing connections prematurely.

Note

Do not reuse the same DirectorySynchronizer object. Each object can be used only once, hence, a new object must be created for the retry.

SyncConflictStrategy syncConflictStrategy() const

The strategy to be used in case a sync conflict is detected.

This determines how the synchronizer proceeds in case it detects a sync conflict.

void setSyncConflictStrategy(SyncConflictStrategy strategy): Set the strategy to be used when a sync conflict is detected.

SynchronizerFlags flags() const

Settings to fine tune the synchronization.

This returns the flags which control some aspects of the sync. By default, this is set to SynchronizerFlag::DefaultFlags.

void setFlags(const SynchronizerFlags flags): Set the flags which control some of the behavior of the sync.

SynchronizerState state() const

The state of the synchronizer.

This returns the current state of the synchronizer. After creation, the synchronizer is in the SynchronizerState::Ready state. In this state, the start() method can be called to trigger the synchronization. This causes the synchronizer to change into the SynchronizerState::Running state. As soon as the sync is done, the synchronizer changes into the SynchronizerState::Finished state.

SynchronizerError error() const

Indicates the status of the finished synchronization.

This returns the error reason for the sync. If the sync was successful, this is SynchronizerError::NoError. Otherwise, this is set to a value indicating the reason why the sync was not successful.

QString errorString() const: Get a textual error description for the last error.

void start()

Start the synchronization.

This starts synchronizing the configured local and remote folders. Calling this will cause the synchronizer to change into the SynchronizerState::Running state. Eventually, the synchronization will finish. This is indicated by emitting the finished() signal. It will then switch to the SynchronizerState::Finished state.

Note

Calling this method has no effect if the synchronizer is not in the SynchronizerState::Ready state.

void stop()

Stop the running synchronization.

This method can be used to stop the running synchronization. This will cause currently running jobs to be stopped. The overall sync will then be finished with a SynchronizerError::Stopped error.

Note

Calling this method has no effect, if the synchronizer is not in the SynchronizerState::Running state.

Signals

void finished()

Synchronization has finished.

This signal is emitted to indicate that the sync has finished - independent on whether it was successful or not. Check the value returned by the error() function to learn if the sync finished with a problem or not.

void logMessageAvailable(SynchronizerLogEntryType type, const QString &message)

Used to report messages during the sync.

This signal is emitted when the synchronizer runs a particular action or encounters an issue. The type indicates the type of message, e.g. if it is an informational message, a file is being downloaded or a remote folder being created.

Depending on the concrete type, message is either an arbitrary string (containing more details about the issue) or the path (relative to the local and remote root folder) which is being affected.

void progress(int value)

Indicate progress of the sync operation.

This signal is emitted from time to time to inform about the overall progress of the sync operation. A negative value indicates unknown progress (this might be reported in the initial phase when the sync plan is being created). As soon as the known steps are gathered, this signal is emitted with values between 0 and 100, indicating the overall progress of the operation.

Protected Functions

explicit DirectorySynchronizer(DirectorySynchronizerPrivate *d, QObject *parent = nullptr): Constructor.

The SynqClient::SynchronizerError enumeration is used to encode the various errors that might occur during the sync.

enum class SynqClient::SynchronizerError : quint32

Used to encode the type of error during synchronization.

This enumeration holds the various types of errors that might occur during a synchronization between a local and a remote folder.

See also

DirectorySynchronizer

Values:

enumerator NoError: The sync was successful - no error occurred.

enumerator Stopped: The sync has been stopped programatically.

enumerator MissingParameter: Indicates that some properties required for the sync are missing.

enumerator InvalidParameter: Indicates that some properties have invalid values set.

enumerator FailedOpeningSyncStateDatabase

Opening the sync state database failed.

This error is reported when the synchronizer was not able to open the sync state database when starting the sync.

See also

SyncStateDatabase::open()

enumerator FailedClosingSyncStateDatabase

Failed to close the sync state database.

This error is reported if at the end of the sync, closing the sync state database failed.

enumerator FailedCreatingRemoteFolder

Creating the remote folder has failed.

This error indicates that an error occurred while creating the remote folder. Creating the remote folder is done when the synchronizer detects we are doing the very first sync.

enumerator FailedCreatingLocalFolder: Creating a local folder has failed.

enumerator SyncStateDatabaseLookupFailed

Looking up entries from the sync state database failed.

This error indicates that looking up entries from the sync state database failed.

See also

SyncStateDatabase::findEntries()

enumerator SyncStateDatabaseWriteFailed: Writing to the sync state datbase failed.

enumerator SyncStateDatabaseDeleteFailed: Deleting entries from the sync state database failed.

enumerator FailedListingRemoteFolder

Listing a remote folder failed.

This error indicates that listing the entries in a remote folder failed.

enumerator FailedDeletingLocalFile: Deleting a local file has failed.

enumerator FailedDeletingLocalFolder: Deleting a local folder has failed.

enumerator Stuck

The sync got stuck.

This error indicates that during the sync no further actions could be started due to unfulfilled dependencies.

enumerator UploadFailed: Uploading a file failed.

enumerator DownloadFailed: Downloading a file failed.

enumerator WritingToLocalFileFailed: Writing to a local file has failed.

enumerator OpeningLocalFileFailed: Opening a local file failed.

enumerator FailedDeletingRemoteResource: Deleting a remote resource has failed.

enumerator FailedListingLocalFolder

Listing a local folder failed.

This error indicates that the synchronizer could not reliably list a local folder and therefore could not safely decide whether previously synced resources were deleted.

enumerator SuspiciousRemoteDeletion

The sync plan contains a suspicious amount of remote deletions.

This error is reported when the synchronizer would delete nearly all known remote resources in a RemoteWins sync. This is treated as suspicious because it can be caused by a broken local directory listing.

The SynqClient::SynchronizerState is used to represent the states a synchronizer runs through.

enum class SynqClient::SynchronizerState : quint32

The states of a synchronizer.

This enum encodes the states a synchronizer runs through.

Values:

enumerator Ready: The synchronizer is ready and can be started.

enumerator Running: The synchronization is currently running.

enumerator Finished: The synchronization has finished.

In case a sync conflict occurs, the SynqClient::SyncConflictStrategy enum is used to determine how to proceed.

enum class SynqClient::SyncConflictStrategy : quint32

Determines how to proceed in case a sync conflict is detected.

During a sync operation, it might happen that a sync conflict is detected. A sync conflict happens if a file is modified both locally and remotely.

This type is used to instruct the DirectorySynchronizer how to proceed in case of such a conflict.

Values:

enumerator RemoteWins

Use the version of a file the remote provides.

Use this strategy if you want remote changes to get precedence over local ones. Using this strategy, upon a conflict the remote file will be downloaded and local changes be replaced.

enumerator LocalWins

Use the local version of a file.

If this strategy is used, upon a conflict the local version of a file is used and uploaded to the remote.

Some aspects of the synchronization can be controlled by passing a SynqClient::SynchronizerFlags value to the synchronizer:

typedef QFlags<SynchronizerFlag> SynqClient::SynchronizerFlags

Flags used to fine tune exec execution of a sync.

See also

SynchronizerFlag

The valid flags are encoded in the SynqClient::SynchronizerFlag enum:

enum class SynqClient::SynchronizerFlag : quint32

Fine tune execution of a sync.

The values in this enumeration are used to fine tune the behavior of the synchronization.

Values:

enumerator NoFlags

Used to indicate “no” flag.

This can be used to create an empty flags value. It has no option turned on.

enumerator CreateRemoteFolderOnFirstSync

Create the remote folder on the first sync.

If this option is set, the remote folder will be created upon the first sync if it does not yet exist.

enumerator AllowMassRemoteDeletion

Allow sync plans that delete nearly all known remote resources.

By default, RemoteWins syncs reject suspicious mass remote deletion plans as a safety guard. Set this flag to allow such plans.

enumerator DefaultFlags

Default flags used for synchronization.

This is the default set of flags used for synchronization. It includes the following list of flags:

CreateRemoteFolderOnFirstSync

Log messages produced by the synchronizer use the SynqClient::SynchronizerLogEntryType enumeration to encode the concrete type of log message.

enum class SynqClient::SynchronizerLogEntryType : quint32

The type of log message made by the DirectorySynchronizer.

This enum is used to classify the type of log message emitted by the DirectorySynchronizer class during a sync.

Values:

enumerator Information

An informatinal message.

These messages are used to carry arbitrary information about the sync procedure.

enumerator Warning

A warning.

Messages of this type carry an arbitrary string used to warn about an unusual, but non-fatal issue during the sync.

enumerator Error

An error occurred.

A message of this type carries an arbitrary error string used to report a fatal error which causes the sync to stop.

enumerator LocalMkDir

A local folder is being created.

Messages of this type carry the path of a folder which is created locally.

enumerator RemoteMkDir

A remote folder is being created.

Messages of this type carry the path of a folder which is created remotely.

enumerator LocalDelete

A file or folder is deleted locally.

Messages of this type carry the path of a file or folder which is removed locally.

enumerator RemoteDelete

A file or folder is deleted remotely.

Messages of this type carry the path of a file or folder which is removed remotely.

enumerator Download

A file is being downloaded.

Such a message carries the path of a file which is downloaded.

enumerator Upload

A file is being uploaded.

Such a message carries the path of a file which is uploaded.

The SynqClient::RemoteChangeDetectionMode enumeration is used to select the way the synchronizer tries to discover remote changes.

enum class SynqClient::RemoteChangeDetectionMode : quint32

Determines the way remote updates are detected.

This enum is used to select the way remote updates are detected during a synchronization.

Values:

enumerator FoldersWithSyncAttributes

Remote folders report a sync attribute.

This mode is used e.g. for WebDAV. The assumption is, that the server reports sync attributes also for folders. Whenever a file in that folder or in a sub-folder changes, the sync attribute of that folder changes. This way, the sync can build the list of remote changes by querying the remote root folder. If the sync attribute of that folder changed, we know that some file inside this folder changed, so the sync proceeds to check the files and sub-folders. This procedure is done recursively, to find all remote changes.

Note that this modes also works if no sync attributes are reported for folders - however, in this case the sync will have to scan all folders each time a sync is run. This means that the minimum number of queries grows with the number of folders that need to be synchronized. Hence, if a backend does not provide sync attributes on folders, ideally an other mechanism is used to discover remote changes.

enumerator RootFolderSyncStream

The remote provides continuous updates recursively for the root folder.

This mode is used for e.g. Dropbox. In this case, the sync assumes that the remote root folder can be queried recursively and that a cursor is returned such that the next time, only the changes since the previous sync can be queried from the server.