Full Sync Architecture Questions


#1

Hello, I was able to configure Sync Realm successfully but after days of trying to wrap my head around how to architect my realm I’m coming to this forum for help.

According to the documentation, full sync is now the recommended way to use Sync Realm.

  1. I want to organize my data just like in the documentation where I would have On-Demand and Common Realms. If we take the To Do app as an example, I would have a common realm that would store user info, and on demand-realms for each user to store their lists, tasks etc. For the latter, I think it’s pretty straightforward – I just need to use user specific realms. However, for the global Realm, how would I go about letting users store their personal info upon registration once, and not have access to it after that? I can’t make them admin, because they would then be able to modify other users’ data. Do I need to create an admin account and use my personal server to register their info on their behalf? Is there an easier way to do this?

  2. Is there a way to tie the user info entry to the user with full-sync realm?

  3. Finally, say I want to have shared to-do lists between users. How would I go about implementing that using full sync?

Any guidance would be greatly appreciated.

Thanks.


(jay) #2

Can you clarify? What data would be stored once and not accessed again?

Is there a way to tie the user info entry to the user with full-sync realm?

If each user has their own Realm as suggested in your question, what other tie would there need to be? Are you asking how to know what To Do data goes with what user? It appears from your question (for this use case) you would not need a common or on-demand realm. Each user has thier own realm file which stores all of their user info as well as their To Do’s. e.g. Neither the user info nor their To Do’s are common - it’s specific to just that user.

Finally, say I want to have shared to-do lists between users. How would I go about implementing that using full sync?

Ah - this is where a common Realm may come into play. There’s a variety of ways to implement this but each user (in their own realm) would have a list of other users primary keys (uuid for example) of data they could access in the ‘common’ realm.

It’s kind of a vague question though as we don’t know what kind of security you want to implement or what the other parameters are. Technically since each user has their own realm (which is stored locally and synced to Realm Cloud) you could just add users to your own Realm that you want to have access to it - that would eliminate a common Realm.


#3

Hi Jay,

User info data such as: email address, first and last name, etc… To be clear, I would like to store this data in a global realm just like what is done in the documentation for ease of access in case I need to look up if an email address already exists, query all the email addresses etc etc.

I was referring to tying each global realm user info entry to its corresponding user.

So if users A and B share list C. List C should remain in the common realm, and its primary key should be stored in the database of users A and B. Am I understanding this right?

If I want to grant write access to users A and B to list C in the common realm, how would I do that? Use an admin account server side to grant access every time a shared list is created?

Hope I’m making sense.
Appreciate your help.


(jay) #4

This is going to be tough to answer as there are many possibilities.

Let me outline two.

Each user has their own discreet Realm for the To Do list and a common Realm that stores all user data. The common Realm will be needed to more easily perform queries across user accounts. The Realm list would look like this

/Users
/user_0_realm
/user_1_realm
/user_2_realm

If user_0 wanted to share the data in their realm with user_1 and user_2, they would Offer Permission to those users to share the data in their Ream. You can offer read or read/write access. That should be fairly straightforward.

Keep all data within one realm. This makes any query for data very straightforward. The Realm would look like this

/All_Data

then within the All_Data realm there would be tables.

Users
To_Do

The users table contains User Objects with email address, screen name etc.

The To Do table would be the ToDo’s with each To Do being linked back to the user that owns it and then another property for other users that can access it

So for example with user_0 logs in, they query for All To Do’s that they own and then a separate query for all To Do’s that are shared with them.

Again, the advantage here is queries are seemless and all data is easily shared. The downside is that it may be less secure since all users have access to all data. Another factor is quantity of data - a couple hundred users with some data would not be an issue - however, millions of users would overwhelm this config (for now, until Realm is backed by MongoDB then it may be a different story).

I think the ultimate solution will be a hybrid between those two options that maximizes performance and scaleability while maintaining security.


#5

This is very helpful. Thank you.

I think implementing architecture 1 makes more sense to me. Inside the global /Users realm (which would be a table with columns like email, first name, last name etc…), the only way I can think of to tie each entry to its corresponding user is to add a field for userIdentifier or duplicate the data inside the user-specific realms.


(jay) #6

Tying everything together would be done in a similar fashion as SQL - you would have an external key that ties users to their data.

So the user object would have a primary key of user_id, that would/could be created with UUID.string which guarantees a unique user id for each user.


#7

That makes sense.

If I am using the admin user to write the user info on the registered user’s behalf (for security reasons global realm /Users would only grant written access to admin), wouldn’t the following code generate the admin user’s primary key instead of the registered user’s?

@objc dynamic var userID = UUID().uuidString
  override static func primaryKey() -> String? {
    return "userID"
  }

In this case I would probably have to set the value of userID manually?


(jay) #8

Well, kinda. To add other ‘users’ you would need to do it though the Realm Studio App for the account as well as with code as you mention. However, your admin user would already exist in Realm Studio so you could use that to allow other users to access data by inviting them.

See Working With Users as well as Access Control

That being said… Realm does not have a robust user authentication (IMO) and handling system so you may want to consider another user authentication system.

See SyncCredentials for further info.


#9

Got it. Thanks a lot, @jay.


#10

@jay Apologies for the many questions. I implemented architecture 1 and I’m seeing some issues.

Locally
Data Model -> List, Task, and User classes.
default.realm -> Contains a Task and a List class. Preloaded with data.
global.realm -> Contains a User class for storing user info.

In the cloud
/global -> corresponds to the local global.realm. Created it manually in the cloud and gave all users write access.
/~/newRealm -> Upon user registration the content of default.realm gets uploaded to the user-specific realm.

Problem
When a user registers, the data is synced correctly except for the user-specific realm which contains an empty class User on top of the List and Task classes despite specifying object types in the configuration.

This is how I’m trying to map out my realms in the cloud:
default.realm <-> user-specific realm
global.realm <-> global.realm

But instead both default.realm and global.realm classes are showing inside user-specific realm. I hope I’m making sense.

Code

var syncDefaultConfiguration = syncUser.configuration(realmURL: url, fullSynchronization: true, enableSSLValidation: true, urlPrefix: nil)
syncDefaultConfiguration.objectTypes = [List.self, Task.self]
                                        
var syncGlobalConfiguration = syncUser.configuration(realmURL: userUrl, fullSynchronization: true, enableSSLValidation: true, urlPrefix: nil)
syncGlobalConfiguration.objectTypes = [User.self]

 let syncRealm = try! Realm(configuration: syncConfiguration)
 let syncUserRealm = try! Realm(configuration: syncUserConfiguration)

(jay) #11

Yes, you’re making sense. It’s going to be hard to really troubleshoot this via this forum but I do have one question; before the code in your question, is any Realm or Realm objects being accessed? In other words, I ran into a similar situation where I had one realm with one class and another realm with a different class but both Realms kept getting both classes. That was because of where in (in the code sequence) I accessed the default realm, which instantiated those classes and also instantiated them in the other realms.

Oh - are you using any kind of singleton pattern for Realm access or is it all done within a function where the vars go out of scope on completion of the function?


#12

I’m using the latter.

I played around with it a bit and modifying my code a bit helps. You are correct, it really depends on where I’m accessing realm. Still looking into it.
I’m also dealing with a weird issue where when I log in with two different devices with the same account, the data gets duplicated for some reason. Both devices access Realm Sync with the same credentials but each have their own independent entries in the database’s table. It’s weird.

Example
User A registers on device 1 -> user info get saved
User A logs in on device 2 -> querying user info returns nil even though data is there.

Or if I create an object on device 1, it’s not showing on device 2 even though it’s the same user.

Flow

  • Inside App Delegate’s didFinishLaunchingWithOptions I open local default.realm to perform migrations etc and make it the default realm config.
  • Then, I take the user to a login/register page and I open the different realms in this order: default.realm -> global.realm -> global sync realm -> user-specific sync realm.
  • Finally, upon login, I take the user to the MainViewController where lists are displayed. I open global and user-specific sync realms to display the data.

Do you see anything obvious that I’m doing wrong? Could it be the schema version not being set for sync Realm? (That’s the only thing I could think of) I think using a singleton to access the different realms is a very good idea. I’ll work on that.


#13

I still haven’t been able to solve the sync issue I’m seeing. I filed a ticket and haven’t received a response – it’s been over a week. Can anybody steer me in the right direction?

How long does support usually take to reply to a ticket?

To reiterate, this is the issue I’m seeing: I have 2 bundled local Realm located in a Resource folder. I created a user-specific Realm (for each user) and a global Realm. When I log in with 2 different devices (with the same user), each device is managing different entries of the user-specific realm tables – even though it’s the same table.


(jay) #14

I have users that are accessing Full Sync’d Realms from different devices and am not seeing that issue. For your situation, however, the terminology and architecture being used isn’t clear as to what it means. For example

bundled local Realm

If it’s ‘bundled’ then it’s included in the App as a resource (as in it’s showing in XCode) and is not modifiable and must be local as it’s part of the app.

On the other hand, a local Realm would be… a Realm file that’s stored locally on the device only and not sync’d with Realm Cloud.

Then there’s a User Specific Realm and a Global Realm. I am guessing the user specific Realm only has content for that user and the Global is shared by all users? Are these both Cloud Sync’d realms? If so, Full Sync or Query (partial) Sync?

Then

I open local default.realm

Which of the above is the default realm? the local? User Specific? Global or a 4th Realm that’s something else?

I open the different realms in this order: default.realm -> global.realm -> global sync realm -> user-specific sync realm.

Why is there a global realm and a global sync realm? Isn’t it data shared by all users? What’s the difference?

What I am illustrating here is that only you know your app and we don’t know your architecture or code - without minimal, verifiable code that duplicates the issue, its going to be hard to troubleshoot as you could simply have a var being set to the wrong realm.


#15

Makes sense. I’ll try to make it as clear as possible:

Context: My app has been out for some time now and I’m persisting users’ data using Realm Database. I’m trying to implement Realm Cloud along with authentication in the next version.

Problem: When my users install the next version of my app, I would like to migrate their data into the cloud upon sign up and have them use Realm Cloud.

Pre-cloud implementation configuration: I currently have a local Realm database, called default.realm. The default.realm is part of the Copy Bundle Resources, and is preloaded with app on-boarding data.

Cloud implementation – What I’m trying to implement
My goal is to implement a full sync configuration, per the documentation’s recommendation.

Locally
Data Model -> List, Task, and User classes.
default.realm -> Contains a Task and a List class. Preloaded with data.
global.realm -> Contains a User class for storing user info. (I created this to store user info locally)

In the cloud
/global -> corresponds to the local global.realm. Created it manually in the cloud and gave all users write access.
/~/newRealm -> Upon user registration the content of default.realm gets uploaded to the user-specific realm.

Relationships between local and cloud Realms
default.realm <-> user-specific realm
global.realm <-> global realm

I created a global.realm locally to have it sync with the /global/ realm in the cloud. On the other hand, default.realm is what my users are currently using, and I want to have it sync with the user-specific realm in the cloud.

Maybe my understanding of the relationship between Realm database and Realm sync is what’s lacking here. But since I’m implementing a full sync architected around one global and multiple user specific realms, I thought I needed to have to separate files locally.


#16

Adding some code:

  • In my AppDelegate.swift, I’m configuring my default.realm (local):
       let config = Realm.Configuration(
            schemaVersion: 106,
            migrationBlock: { migration, oldSchemaVersion in
                // migration code
        })
                
        Realm.Configuration.defaultConfiguration = config
        
        let defaultPath = Realm.Configuration.defaultConfiguration.fileURL!.path
        if let v0Path = Bundle.main.path(forResource: "default", ofType: "realm") {
            do {
                try FileManager.default.copyItem(atPath: v0Path, toPath: defaultPath)
                print("Copied.")
            } catch {
                print("Did no copy.")
            }
        }
        print(defaultPath)
    }
  • My reference Realm file. I’m using global variables to make it easy to call realm across my app:
import RealmSwift

var DefaultRealm = ReferenceRealm.shared.realm
var GlobalRealm = ReferenceRealm.shared.globalRealm

private class ReferenceRealm {
    
    static let shared = ReferenceRealm()
    
    lazy var realm: Realm = {
        var syncConfiguration = SyncUser.current!.configuration(realmURL: Constants.REALM_URL.appendingPathComponent("/~/newRealm"), fullSynchronization: true)
        syncConfiguration.objectTypes = [FoldersPointer.self, Folder.self, FolderItem.self, ArchivedNoteList.self, CompletedTask.self, CustomListsPointer.self, DefaultListPointer.self, Habit.self, HabitList.self, Note.self, NoteList.self, PinnedListsPointer.self, TaskList.self, Section.self, Task.self, Subtask.self]
        syncConfiguration.schemaVersion = 106
        let realm = try! Realm(configuration: syncConfiguration)
        return realm
    }()

    lazy var globalRealm: Realm = {
        var syncConfiguration = SyncUser.current?.configuration(realmURL: Constants.REALM_URL.appendingPathComponent("/global"), fullSynchronization: true)
        syncConfiguration!.objectTypes = [User.self]
        syncConfiguration!.schemaVersion = 1
        let realm = try! Realm(configuration: syncConfiguration!)
        return realm
    }()

}
  • Upon login/Sign up code:
if let syncUser = syncUser {

     DispatchQueue.main.async {
                                        
             defer {
                    // Going to welcome screen
                     self!.goToWelcomeViewController()
             }
            
            // Realm
            // This is my local default.realm                         
            let realm = try! Realm()
            // Accessing+Configuring global and user specific realm (DefaultRealm)             
            let syncRealm = DefaultRealm
            let syncUserRealm = GlobalRealm
            // If user signed up, copy local realm data to user specific sync realm, and save user info                     
            if syncUserRealm.object(ofType: User.self, forPrimaryKey: syncUser.identity!) == nil {
                    try! syncRealm.write {
                           // Copying data (not including code since not relevant)                     
                    }
            }
     }
}

Please let me know, if you need to see any other portion of code.


#17

Almost 2 weeks and still no response from support? I don’t get it… Will someone ever get to my ticket or should I just give up? If you are too busy to get to it at least provide me with an ETA but leaving me hanging when I’m stuck with the release of my app’s next version is the worst.

Being Processed since 14 days 11 hours


(jay) #18

That’s a long time to wait…

Just trying to help here so excuse the questions. I’ve read and re-read your descriptions and the naming conventions lead to a bit of confusion and how your realms are being used is unclear (to me).

Problem: When my users install the next version of my app, I would like to migrate their data into the cloud upon sign up and have them use Realm Cloud.

What is ‘their data’? the List, Task and User Class data? Or just some of that or all? What should be shared amongst users? Anything? Or does each user only have 100% their own discreet data?

global.realm -> Contains a User class for storing user info. (I created this to store user info locally)

There’s a global realm that stores data locally? Why call it global if it’s local? And why global if it’s only for this users data, locally? I see that the new global realm stored in the cloud will allow write access from all users. Does that mean they share data between users or is their data only their data? Just curious

I currently have a local Realm database, called default.realm. The default.realm is part of the Copy Bundle Resources, and is preloaded with app on-boarding data.

While that is called default.realm, it’s not really acting as a default ream because it cannot be written to as it’s part of the bundle. Are you copying that data to another realm file called default.realm that’s actually a realm file which can be read and written to? If that’s not the case, where are new tasks and lists stored?


(jay) #19

To re-visit the question, it’s unclear where the difficulty lies. Let me throw out a very basic example and maybe you can clarify where yours isn’t working.

Suppose we a task manager project that exists on one device. So here’s the Task class

class TaskClass: Object {
    @objc dynamic var task_id = NSUUID().uuidString
    @objc dynamic var task = ""
    
    override static func primaryKey() -> String? {
        return "task_id"
    }
}

and in the UI we can read, write and edit tasks. All basic stuff. All of the tasks are stored in a local Realm file called Tasks.realm and is configured in the viewDidLoad method of the main viewController.

override func viewDidLoad() {
    super.viewDidLoad()
    
    var config = Realm.Configuration()
    config.fileURL = config.fileURL!.deletingLastPathComponent().appendingPathComponent("Tasks.realm")
    Realm.Configuration.defaultConfiguration = config

Now we (as you do) want to expand the app so it’s usuable on multiple devices - Realm Cloud (or whatever they decide to call it) is an answer.

So in my project I add some code that will enable it to talk to my Realm Cloud and my Realm is called CloudTasks.

So I set up (per the docs) some constants

struct Constants {
    static let MY_INSTANCE_ADDRESS = "coolness.us1.cloud.realm.io"
    
    static let AUTH_URL  = URL(string: "https://\(MY_INSTANCE_ADDRESS)")!
    static let REALM_URL = URL(string: "realms://\(MY_INSTANCE_ADDRESS)/CloudTasks")!
}

and then a Singleton to access it (yeah yeah, singletons are evil in Swift - don’t get me started).

var RealmService = MyRealm.shared.realm

private class MyRealm {
    static let shared = MyRealm()
    
    lazy var realm: Realm = {
        let syncUserConfig = SyncUser.current?.configuration(realmURL: Constants.REALM_URL, fullSynchronization: true)
        let realm = try! Realm(configuration: syncUserConfig!)
        return realm
    }()
}

Then, when the user logs in I present a dialog asking if they want to migrate their data to CloudTasks, and when they do, here’s the code

func migrateLocalDataToCloud() {
    do {
        let realm = try Realm()
        let taskResults = realm.objects(TaskClass.self)
        
        let realmCloud = RealmService
        try realmCloud.write {
            for task in taskResults {
                realmCloud.create(TaskClass.self, value: task, update: .all)
            }
        }
    } catch let error as NSError {
        print(error.localizedDescription)
    }
}

From then on, all data is written to the could with the RealmService. This code works for Realm objects and assumes you are using primary keys across the board and are handling any oddball backlinks separately.

There are several ways of doing this same task but this is probably the simplest example.

So now you have that, where’s the difficulty?