Efficient handling of offline deletions using Data Connectors

Ask the community
1

Hi,

I have an application that is developed with offline support, which means that all information is temporarily stored in runtime classes on the user’s device. The data is only sent to the server when the user actively chooses to submit the report and has sufficient internet connection.

When submitting data to the server, the runtime classes are persisted, but only if their Object State is marked as either “New” or “Updated”.

Regarding data deletion, the application follows a different process. When a user deletes rows while offline, these are not immediately deleted from the server due to lack of internet connection. Instead, they are stored in separate runtime classes called “Deleted items”. This is done to handle the deletions when the internet connection is restored.

When the internet connection is restored and data needs to be synchronized, the system runs a “Delete filtered selection” against data connectors where the ID in the data connector exists in the class holding the deleted rows. The system identifies which elements to delete by looking at the IDs stored in the “Deleted items” classes. To perform these deletions, “All Objects” is activated on the data connector to access any rows that need to be deleted.

Data connectors are used because they have no performance impact, as they do not hold data and are never “re-evaluated and calculated”, but my question is whether this method of deletion is time and resource-intensive.

Does it matter that “All Objects” is activated for the Data connector regarding this way of deletion? Will “Delete filtered selection” through a data connector potentially be a bottleneck if it has to “iterate” through houndreds of thousands of rows on the server to determine which elements should be deleted? Would it be more efficient to delete individual elements directly by iterating over them and using “Delete Object” where ID equals? Alternatively, modify the data connector to filter on IDs found in the runtime class “Deleted items” to limit data access.

Another approach would be to use an async service to perform the deletion, while the application just registers the elements as “Ready for deletion”, but first, I want to get clarity on the resource usage in the issue described above.

image
image541×217 10.6 KB

image
image430×167 9.54 KB

image
image1031×204 21.3 KB

Best regards,
Sondre

2

Hi Sondre,

I believe the setup that you have now, where you delete filtered in a Data Connector is the best way of doing it. When doing it this way, your client generates a filter and sends that to our servers for deletion.

By doing one-by-one, youll have to establish a conneciton to our servers and database for every object, which will be slower and more resource intensive.

The only benefit, that I can see, of doing one-by-one is that you can add som better error handling, incase only one object fails to delete.

Adding a filter on the Data Connector to only include the IDs of the objects in the To be deleted-datasource could be a nice safeguard, to ensure that if you accidently select Delete All, not all objects will be deleted. But this should not impact performance either.

Pure performance wise, I think the setup you have now is good :slightly_smiling_face:

// Erik

2 Likes