We have a big amount of data to insert into a realm database. The data are coming from several webservices for a total amount of 100Mb to 1Gb.
We need to store each result from network with a transaction. If all the webservices are downloaded successfully, after a post-process we commit the database.
Due to the fact that realm for that transaction requires to work on the same thread and that the webservice are asynchrone, the problem we are facing is that we are not able to call the same thread after a webservice result, how can we manage to handle this situation?
- We cannot load all the data in memory -> the huge amount of data is putting at risk the memory exception
- Storing the result of webservice response in files are not very efficient -> we did this but the performance are bad (we spend 30% of processing in this part)