This forum has moved, please join us on github discussions. We will keep these old posts available for reference. Thank you!

Hidden fields in schema & content access based on role

Hello,

As a follow-up on this thread (I was considering deploying lots of instances of Apostrophe, one per customer, each of whom would have many customers), I’ve learned a good bit about Apostrophe in the mean time, and it looks as though I may have been massively overcomplicating what was necessary, and can instead do everything from a single instance / single DB.

I’m currently pursuing the path Tom mentioned of pivoting everything by siteId. This is promising, but there are two things I am still trying to suss out:

First, I am currently adding siteId in the Piece’s construct beforeSave method. This works as desired:

construct: function(self, options) {
  self.beforeSave = function(req, piece, options, callback) {
    piece.siteId = '5aa82645e1f36bb8032d5e31';
    return callback();
  };
},

I tried adding ‘hidden’ to the schema for this field, just to see if there might be an undocumented feature, but of course this didn’t do the trick. Is there a way to prevent a user from being able to see a field? I could hide the field with css targeting .apos-field-siteid, but I’d prefer something more programmatic / less hacky.

This field is something that only the application logic will ever use, and it’s better if the user isn’t even aware of it, (ideally) even if calling content via the API, though blocking it from the UI would suffice for the time being.

Second, grouping users based on siteId

I’ve done some digging here, but since the question of permissions is such an intricate one, I thought I’d just start by asking for some background. My initial thinking is structuring this something like:

'apostrophe-users': {
  groups: [
    {
      // the siteId
      title: 'manager-1234',
      // a new type of 'permission', which means the user can edit pieces / pages with
      // site id of 1234
      permissions: [ 'edit-1234', 'etc...' ]
    },
  ]
}

on the user doc then you would find:

{
  _id: "c2s3d4g567",
  username: "tom",
  groupIds: ['manager-1234'],
  type: "apostrophe-user"
}

Then, each time a new customer is added to my product we:

  1. Create a new group called “manager-theirsiteid”, and a new permission “edit-theirsiteid” which lets them edit pieces / pages with the associated Id
  2. Add the associated groupId “manager-theirsiteid”

So, my questions are:

  1. Am I thinking about this correctly?
  2. What am I missing?
  3. What functionality in which modules do I need to consider (I’m guessing apostrophe-users and apostrophe-groups are at play here)
    • Do I make a my-apostrophe-users and a my-apostrophe-groups module and create my own ensureGroup function, for example, basically reimplimenting your ensureGroup function but with the customizations I need?
  4. To create users via REST API, I suspect I will need to access the database directly, and then call the appropriate methods from self.apos.users (or something similar)
  5. I also want the Users -> Manage Users list to only return users scoped to this siteId.

Thanks for any input.

You should check out the apostrophe-multisite module. A work in progress but we’re using it for a production client, so we’re close to npm publish on it and the general approach is pretty much locked down:

This module does not attempt to merge them all in one database because with Mongo 3.x and WiredTiger there isn’t much of a penalty for independent databases unless you have thousands of them. Much friendlier to a shared environment than Mongo used to be.

If I were required to force all of the sites to share one database, I would intercept requests at the mongodb collection object level by creating my own wrappers for those objects, and give out fake “database connection” to each site that are already wrapped in that way. Because sooner or later queries/inserts/updates that don’t know about your special siteId are going to burn you if you try to do this at any higher level in the system, and it’s not actually any harder. (Which doesn’t mean it’s easy…)

Such a thing could be released as a mongodb-multiplexer module.

Hi Tom,

Ok, will take a look again at multisite. I did after you mentioned it initially, but given the warning about it being a WIP at the moment, I didn’t want to build a production application on something that presented itself as being fairly alpha, and therefore I suspected highly susceptible to breaking changes.

Helpful to know you’re close to publishing. Looks like you haven’t had a commit in a while, mind pushing what you’ve got so I can test out the latest?

Also, I followed-up with mLab after our last discussion, and they gave me permission to share their much more in-depth feedback, and thought I’d ask for your perspective as well:


From mLab:

We understand you may be limited by the CMS but are happy to elaborate.

We strongly recommend against having a database per user because the only thing in MongoDB that has no hard limit is the number of documents in a collection. As such, truly scalable architectures involve a single database with a small fixed number of large collections containing one or more documents for each user. Because of these limitations, having one database per user creates an artificial ceiling on the number of users you can support with a given deployment.

You will also likely run into problems well before you reach any hard limits. For example, a large number of databases also makes it more complicated when you want to migrate or scale as you grow. Some specific issues are:

Creating a new index for a new or previously unindexed feature requires coordinating that index build across many collections.

Our Telemetry service and UI will not show certain statistics over ~25 database because it simply takes too long to gather them all and it puts undue load on the deployment.

A single “hot” user can overwhelm the resources of a deployment at the expense of other users, and will need to be migrated (with downtime) to its own deployment.

Adding too many databases to a single deployment can reach hard limits like the number of file descriptors (https://docs.mongodb.com/manual/reference/limits/ and https://docs.mongodb.com/manual/reference/ulimit/#recommended-ulimit-settings), you will need to create an additional deployment for additional databases and potentially migrate (with downtime) databases off of the original cluster.

At higher scales, you may be able to shard specific users’ databases (https://docs.mongodb.com/manual/sharding/) but it will be very difficult to shard the overall system. Instead you will be locked into a manual partitioning like manually moving databases between deployments.

The overall difficulty of migrating databases between deployments leads to un-equal use of resources over time, which is cost-inefficient.


As a follow-up I asked: “…instead of “one database per user”, for my use case, it is more accurate to say ‘one database for each of my customers’, and each customer will have many users.”


They replied:

…By users we did mean your customers. However, our concerns do apply moreso if each (customer) database would need separate collections for each of the customer’s users.


So, does this affect your thinking at all? If not, then I’ll defer to your judgement given your intimate knowledge of your product (as well as testing out multisite), and it would certainly be easier for me to just spin up a new instance of Apostrophe per customer.

The multisite repo is up to date. It’s been pretty stable since we made a good call and kicked node-http-proxy out of it.

mlab isn’t wrong but the things they are saying become relevant for some number of databases n, where n is a pretty big number, in the hundreds at least. That’s when it would make sense for a mongo-multiplexer module to be created.

That module doesn’t have to be apostrophe specific so there would probably be a lot of demand for it. It’s not something we have an urgent need for here right now, although it would make a great enterprise support project if you wanted to sponsor it.

1 Like