[R&D] Peer to peer sync capabilities

[fredhersch]

Discrete R&D project to explore how to provide peer to peer sync capabilities which would augment and further strengthen the offline use cases where applications using the SDK are likely to be deployed

Outcomes of the project:

• Design of peer-to-peer sync approach
• Impact assessment on current core-sdk sync strategy
• Recommendations
• Proof of concept

[jingtang10]

from @mberg in #275:

In the context of CHW program in Liberia, we may have users who work offline for months at a time. In this context, the way they sync data is via P2P with a visiting supervisor who downloads new activity from their device which is then submitted to the server when the supervisor gains access to the Internet.

We would want to update the content of the app. Eg. Questionnaire changes. We assume this can be just part of the modified resources that would be part of the P2P sync.

from @pld in #275:

This is the p2p library OpenSRP currently uses, https://github.com/OpenSRP/android-p2p-sync

[ekigamba]

@maanuanubhav999 Can you update this issue with a quick summary and links to any related documents for this? I see the other issue was closed.

[maanuanubhav999]

@ekigamba sure. thanks

[maanuanubhav999]

We have two APIs that seem like a suitable candidates - both have their set of pros and cons.
wifip2p
nearby connection

why not use a library
All the libraries I have encountered were not documented properly or were not updated
Also, there have been major changes in the API related and older libraries could cause some error or misconfiguration.
Related libraries are mentioned in the Document below.

Document regarding the same is attached below, feel free to comment if there is an issue.
Document

@ekigamba

[fredhersch]

Thinking about the discussion to provide a sync_status per resource and the impact on the current db.

I was wondering if it made sense to add this as an additional table within the SQLite DB that would store the local_resource_id with a server_sync_flag and any additional meta_data (as a json field) that needs to be tracked.

What are the pros and cons to this? Please add

Pros:

• Doesn't impact the existing data model
• Flexiblity to add parameters within the meta_data or additional columns with db_migrations that don't impact the core db

Cons:

• Additional indexes required for querying (not sure if this is unique to this design)

cc @jingtang10 @pld

[maanuanubhav999]

This is another approach for syncing.

The above-mentioned library works by keeping a record of the last element sent as lastRecordId and we can easily send any new resource with it
the problem arises when we make changes to already existing resource and to tackle this we can just update id of the resource and thus it will be sent the next time.

will have to make changes to the existing resources but am not sure if changing ID can cause any trouble?

pros:

• saves additional indexing

cons

• if id clashes, we may lose data

[pld]

in the OpenSRP p2p library, do we differentiate between synced over p2p and synced to a server?