Inconvenient API


(Chris) #1

Hi, I’m used to using ORM and I find that Realm is very powerful and convenient, however, I find that updating RealmObject very cumbersome. It requires Write() wrapper and I have to make changes inside it, otherwise it will generate an error.

I understand that it requires transaction but I was hoping it follows more conventional method, i.e., just realm.CommitChanges() at the end without requiring realm.BeginWrite() to begin transaction.

In other words, any changes made to RealmObject are just temporary and it doesn’t begin transaction until CommitChanges is called.

I’m sure you know what I’m talking about but bere is scenario to illustrate the problem.

I have many UI options that are stored in Realm. Since it’s a RealmObject, whenever I change any one of them, I have to wraparound Write(). It is very cumbersome as the changes are made in many different places.
One other problem is that I cannot revert all the changes I made easily.
You also recommend making the transaction in one large group for better performance.

To make it easier, what I do is to create a temporary object and store all the changes myself.
When a user confirms the change, I copy the changes I made back to the RealmObject. It’s not very pretty and I think it should be done by Realm.

What I’m proposing is this.

  1. You can continue to use Write() wrapper. I think it’s still useful since it doesn’t need to copy to a temporary buffer to save memory.
  2. If you modify Realm object without Write(), you will store the changes to a temporary buffer instead of throwing an error. And later, if you call CommitChanges, it will start the transaction and store the changes.

I hope it makes sense.

Cheers!


(Chris) #2
  1. (I forgot to add ^^) RevertChanges to revert all the changes made in the temporary buffer.

Cheers!


(Nikola Irinchev) #3

Because Realm is not an ORM, we don’t have temporary storage where we can write changes to the objects. The property accessors directly interact with the database, which is why the write transaction is required. Even if we did have a way to implement it, I can imagine situations where the API will become increasingly awkward - e.g. what happens if a background process changed the data before you call CommitChanges? Should CommitChanges fail or overwrite the data, even though this may not have been your intention? Additionally, how would that work with change notifications? E.g. you change a person’s first name, but a background process changes it to something completely different (being unaware of your changes). At this point, should we update your local object to reflect the state in the database or your temporary buffer? And what if the object gets deleted while the user is filling out the form?

My point is - we know there’s some value in starting transactions implicitly and catering to the “form UI” use case, but we haven’t found a model that aligns with Realm’s ideology of providing intuitive and unambiguous API.


#4

Doesn’t Realm sync use last write wins? We would find it extremely useful to be able to commit or revert changes as @chrisk suggests. We currently have to create our own temporary objects, copy attributes from Realm objects, edit then copy back to Realm. Adds a lot of complexity to the application to achieve the UI business applications usually require.


(Nikola Irinchev) #5

Realm Sync uses it, but the local database is designed to work independently of Sync. As I said, I understand the sentiment and it’s a valid use case, but until we have a compelling API design that handles all corner cases, it’s unlikely that we’ll be able to provide first-class support for it.


(Chris) #6

Hmm… I understand what you are saying because Realm uses “live object”, however don’t you have the same problem when you get the change notification while you are in between “BeginWrite” and “Comit”?
I suppose the object will be overwritten by the latest Comit.

Well,
I guess there is no right answer in realtime application. I’m ok as long as there is a clear rule.

Likewise, my suggestion can be work like “BeginWrite” and “Comit” pair;
whenever you change RealmObject, “BeginWrite” will be implicitly called.
Will it work? And you can then either “Comit” or “Revert” later.

Thanks.


(Nikola Irinchev) #7

Realm write transactions are locking, which means there can be no changes between BeginWrite and Commit. That’s also the reason why we don’t want to make them explicit - if you just start them on property changes, that would mean that other threads will now lock on attempting to write to the Realm.


(Chris) #8

Wow, really? If one app start BeginWrite and it somehow never Commit, another app will be locked forever?
Is this a design feature?

It explains why I often encounter application freezes.
And if it’s is how it works, storing the changes to the temporary buffer makes more sense.
You store them and commit all at once so that you minimize the gap between BeginWrite and Commit.

On realtime application with many users, it will cause so many problems.
Is there a way to work around it? Like timeout?


(Nikola Irinchev) #9

Not sure I understand why it would cause problems? Why are all these users working with the same database? I would expect that on a single physical machine (phone or computer), people would have one app at a time that writes/reads from the database. Am I missing something about your use case that would explain the need to have many people interacting with the same file?


(Chris) #10

Well, I’m using local Realm.Database right now for a couple of reasons.

One is online Realm is not free while in developing. I don’t understand why you removed standalone server option. Could you please bring it back? I was told that there is no change by MongoDB acquisition but this is the first thing that was changed against favor. Anyway, Local Realm seems to allow multiple users and notification also works.

And another is some issues I have using online Realm causing Unity to freeze. I found a workaround for local Realm but not for online Realm.

Thanks.

ps. I started testing Realm because on paper, it looks really good. Having an automatically synced db can help simply the coding a lot of stuff, however, I find that the current Realm is very limiting, forcing me to workaround many problems. My experience thus far is both hit and miss.

Few stuff that I find beside the inconvenience Commit that causes some griefs.

  1. Lack of Enum (this is quite surprising although Realm is a mature product), and custom type support.
  2. Realm often freezes on Write without warning. Perhaps it needs Timeout, or cancelable?
  3. No inheritance support.
  4. Rudimentary Realm Studio IDE. It looks like a project developed over a weekend. Any plans to overhaul the app?
  5. Linq support seems pretty basic.
  6. Cannot access Realm from different threads. (this forces ugly workaround)
  7. No Unity support. Any plan to start staffing soon?

I know I heard from the roadmap that some of them are under work but they are too sketch. I would appreciate if you can shed some lights and possibly ETA.
Instead of pointing to the MongoDB roadmap, it might be better we keep them here with more information.

Thanks.


(Nikola Irinchev) #11

Okay, that’s a lot of topics, but let’s take them in order:

  1. The Realm package contains both the offline and online database. You can use either one depending on your needs.
  2. The change in pricing happened more than a year ago, way before the MongoDB acquisition.
  3. Local Realms allow multiple users, yes, but I still don’t understand what’s the use case of it - how would multiple users sit and collaborate on the same machine? If you do have a use case where many clients work with the same Realm file, then you definitely need to ensure that write transactions are held for as short as possible and that writes happen on background threads to avoid blocking the UI thread while waiting to acquire a lock.

Regarding your griefs:

  1. Enums are not yet supported because they have different meaning in different languages (e.g. Swift enums are vastly different than c#/Java ones and js doesn’t even have the concept). We do hope to provide some support at a point when we feel we have a good and intuitive API.
  2. As mentioned, starting a write transaction needs to acquire a write lock. If you have multiple apps that work with the same file and hold the write transaction for a long time, it’s not surprising that you would observe freezes while the app is waiting for the lock to be released.
  3. Inheritance support is something we would very much like to do and it’s very high in our wishlist.
  4. While Realm Studio is not as powerful as SSMS or similar tools, it was never intended to be more than a tool that allows you to view and do light edits of your data. That being said, if you have specific suggestions on how to improve it, feel free to open an issue on http://github.com/realm/realm-studio.
  5. LINQ support is another thing we would like to improve in the future, but in the meantime you can use the string-based query language to execute queries that would not be supported in LINQ.
  6. The threading model will be reworked as part of the next major release (Core 6) and will relax the strict threading requirements a little bit.
  7. We don’t have immediate plans to support Unity. That is an option we are continuously considering but we’re not seeing enough customer interest to put it at the top of our priority list.

The Realm roadmap is the ultimate source of truth as to when things are planned. Anything written in forums and issues is subject to change without notice and thus is not a good long-term indication of priorities.


(Chris) #12

Hi, thanks for the answer.
My use case is like I said, cost and the freezing issue on Unity.
I plan to move to online Realm at some point.

Ok, although there are many shortcomings, I still like Realm concept. Now that we are under MongoDB umbrella, I’ll have faith that it will reach the goal. I just hope that we get there no later than what we are told on the roadmap.

Cheers!


(Nikola Irinchev) #13

I think there’s a bit of miscommunication here. I can understand that your issues are the cost and the freezing issue on Unity, but I don’t understand why you have many apps working on the same physical file.


(Chris) #14

Realm will cause Freeze on Unity. It will “Play” once but when you “Play” again, it will freeze while trying to unload Realm but it can’t.
Unity cannot unload p/invoked realm-wrappers.dll.

I spend about a week trying to solve the problem but I found a solution that you replace p/invoke calls with memory pointers. However, this technique only works for the local Realm, the Realm.dll that you get when you install Realm.Database but not the version you get when you install online Realm.

Until Realm support Unity officially, I’ll have to use Realm.Database, unfortunately. I hope that explains why I’m using multiple clients on the local realm file.

However, using local realm often creates Freeze problem using multiple clients. This is not the same Freeze cause by trying to unload p/invoked assemblies. Let me call this “thread block” to make the distinction clear.

As you explained, it looks like the block happens when you trying to write(or just holding the handle) to the realm file while another client is also trying to access the same realm file. I understand that the block can happen when two client trying to write at the same time, however, it’s doesn’t seem to be the case. The block happens when the two clients just start subscribing the notification to the same object.

for example,

var token = realm.All<Person>().SubscribeForNotifications ((sender, changes, error) =>
{
    // Access changes.InsertedIndices, changes.DeletedIndices, and changes.ModifiedIndices
});

I can’t think of any simpler example but I can’t figure out why it’s causing the block just subscribing. I haven’t even written anything to it.

Is this a bug? Or am I missing something?

Thanks.


(Chris) #15

Here is a sample chat app I’m testing for your reference.
Please let me know what’s causing the freeze.
Thanks.

https://drive.google.com/file/d/1dx0OjZJn4dY7r6hMbJXinC05gtmxEjTX/view?usp=sharing


(Chris) #16

Is there a chance that the local Realm can work with the multiple clients?

Realm Studio is very rudimentary at its best and I wrote a Excel importer and when Excel file is changes, it write it to the local realm file, hoping that the app that is running in the background notified the changes. But it’s causing the freeze and I can’t get it to work properly.

I think this is a valid use-case for having the multiple clients to the local realm, otherwise, I don’t understand why I’m using Realm. Realm.Database support notification by design but it’s not useful to have only one clients connected to it. (you don’t need to notify yourself) I believe it’s more like a bug and I hope it gets fixed soon.

Thanks for your help and understanding.


(Chris) #17

Ping…

The support is rather non-existent. I’m worrisome working with Realm. Hmm…


(Nikola Irinchev) #18

This is a forum and not a support channel. I’ve answered your questions, but don’t have the time to debug your application. If someone from the community feels like doing it for you, they’re more than welcome to go for it.


(Chris) #19

So, where is the support channel? I was pointed to this place from the support ticket.
I’m seriously getting worried about working with Realm.

Beside, not being able to work with multiple clients on a local realm is rather a bug on Realm and I’m not asking you to debug my application.
So, you are saying it’s how it is and it won’t be fixed?


(Nikola Irinchev) #20

We don’t offer dedicated support for users of the open source database. Realm was designed to be a per-device database and typically you have only a handful of apps operating with the same Realm and very rarely do they run together. That being said, we do support multiple clients working on the same local Realm and if you can demonstrate that there is a bug with that, then file an issue against the github repository for the SDK you’re using.

I created a trivial app that opens a Realm (hardcoded to C:\work\example.realm) and makes changes to it every 2 seconds. I ran multiple copies of that app and they all saw each other’s changes and none of them froze. I’m attaching a video of my experiment as well as the source code of the project I used to record it.

Video

Project