Port apostrophe-localization to 2.0.0

[fotisp]

Hello Tom,

I am in the process of porting the apostrophe-localization module here (https://github.com/punkave/apostrophe-localization) to 2.0.0.

From the looks of it, I can understand the logic, and how you monkeypatch some core methods to provide seamless localization capabilities when the module has been enabled.

I am stuck in two parts and just want to get your input if possible.

First to verify the following: I am replacing superBeforePutPage with self.docBeforeInsert which seems to do what I want, don’t know if there is anything I should on the lookout for

I am stuck in the following two methods:

1)Is 0.5 beforeLoadWidgets the same as apostrophe-widgets/index.js -> load? Should I monkey patch this?
2)What is the 0.5 callLoadersForArea equivalent in 2.0, I couldn’t find anything?

Is there anything you would like to note or a heads up regarding this effort?

Thanks,
Fotis

[boutell]

Hi Fotis, it’s fantastic that you’re working on this.

The docBeforeInsert/docBeforeUpdate/docBeforeSave callAll methods are the right general idea. However you could have some timing issues with this. Other modules can have those methods too and if things don’t happen in the right order it could cause surprises.

Since you need such tight control over when it all happens, you would be better off using the “improve” feature to ship a module that enhances apostrophe-docs, subclassing and for all intents and purposes becoming that module as far as the project is concerned. i.e. like extending it at project level but in an npm module.

For an example of this being done successfully see the apostrophe-pieces-import module (which improves apostrophe-pieces, but same idea). This isn’t about duplicating all the code! Just overriding the bits you need to.

Then you can override the core save and load functionality to do whatever you need it to. And, we’re open to adding some hooks to the core methods (i.e. you could submit a PR) so you can override the minimum necessary.

As for areas, that all happens in an improvement of the apostrophe-cursor type. Look in the areas module and you’ll find a cursor.js that implicitly subclasses that type to add widget loading. But I’m not sure if you’ll need to touch that anyway, as long as you can do the dance of swapping localized versions of the content on load and save via an improvement of apostrophe-docs.

One last note: no, we are not currently in the middle of doing this work ourselves (we don’t currently have a client need for it).

Thanks!

[fotisp]

Ok Tom,

First, thanks for pointing out “improve” I will check that out (any links available to documentation so I don’t keep asking silly questions ?).

I will monkeypatch docsBeforeInsert, this is exactly what I needed.

However, in apostrophe-docs/lib/cursor.js it is not clear to me what I should start overriding for the load method. So if I define an “areas” filter like apostrophe-areas/lib/cursor.js in my own “improve” module, will that “override” the one defined by apostrophe-areas? Can you shed some light.

If I can hack this on a single modules that would be excellent. One last thing, does this need to be an npm module, or are you saying this just for distribution sake?

Thanks,
Fotis

[fotisp]

Hello Again,

So I have made some progress on this. I am able to successfully save docs using a locale which is set through the url or through the session. (Currently I verify that localized exists in mongo)

Now I am at the part where I need to load this information. It is not clear to me where I need to hack the “core load” method you mention. From your original e-mail it seems that this can be done in my custom localization improve module.

Upon further inspection this is what I found in the comments of In apostrophe-docs/lib/cursor.js

// Usually you'll be working with a subclass of `apostrophe-cursor`
// appropriate to a particular type of piece or page. Each subclass
// typically adds new filters. Some modules also add filters to the
// main `apostrophe-cursor` class that are useful with all types
// of docs, like the `areas` filter that calls loaders for
// the widgets in Apostrophe areas.

So my assumption is that apostrophe-areas is the subclass which adds the load filter. In the apostrophe-areas/lib/cursor.js file it seems that the “area” filter has the code I need to mess with. However I was unsuccessful in overriding that functionality with an “improve” module of apostrophe-areas.

Can you point in the right direction here?

FP

[boutell]

Look at:

lib/modules/apostrophe-areas/lib/cursor.js

Notice that in lib/modules/apostrophe-areas/lib/api.js this gets pulled in with:

// merge new methods with all apostrophe-cursors
self.apos.define(‘apostrophe-cursor’, require(’./cursor.js’));

You could do that too. Your new definition of the filter would “win.” It’s implicit subclassing, just like with modules, it’s just a type that’s not a module.

However you have the problem that you’ll want to extend the old definition rather than copying and pasting so much code. And you would be better off anyway intercepting before the areas module adds the area filter.

So I would suggest something like improving apostrophe-docs (which I believe you’re already doing), which also defines apostrophe-cursor (for the first time), and adding an additional definition of apostrophe-cursor there; a new filter (localize would be a good name) that you add there should run its after handler before one added by apostrophe-areas, because the docs module is initialized before all other modules that care about docs, and your improvement of it counts as part of it. It could also swap in the right version of the title property, and so on. This gets us back to the ability to solve this like in 0.5.

But there are still some interesting problems, like how UI filters for various fields are supposed to work across languages. That would make our mongo queries a lot more complex.

It makes me wonder if there should be a separate object in the docs collection for each localization; that would make the localize filter very simple —it would just call .and({ culture: culture }). And your overrides for saving documents might get simpler too. They would just introduce the appropriate culture property.

In this scenario, slugs would have to be distinct for each language (because of the unique index), but that could be desirable.

Falling back to the default language would be a little tricky in this implementation. Or very easy, if you decide that they all get created as needed on save, so they are always in the database even if they are initially just copies of the default language’s content.

What else is tricky in this scenario… joins. They would probably need to continue to point to the default culture’s version of the doc being joined with.

Possibly this could be addressed by changing IDs to look like:

xxxxx-CULTURE

So the right suffix could just be added. But certainly our join loader would need some new logic.

This is a big issue. (:

[fotisp]

Hello Tom,

Thanks for your feedback. I am going to be checking the code here

https://github.com/fotisp/existanze-docs-localized

The code there is by far good to go, but I just want to have something which I can refer to when I run into issues. It is basically the same 0.5 code, copied in the hopefully right places.

Like you said I have been trying to improve the apostrophe-docs module, and I am running into the following problems:

---

1. req.locale

Int the middleware which sets this value you will see that I am using req.session.locale for now, because even though I am setting req.locale to some value, once it reaches docBeforeSave it is always ‘en’ regardless.

2. cursor.js

Here req.locale is not even set. Once apostrophe loads, regardless if there is a request for the site, the localize filter runs. That is why I am hardcoding it.

I know ensureProperties, isArea, and isUniversal is duplicated, I will abstract these once I get it working.

3. ensureProperties
Since this code is run in cursor.js it seems that it is not necessary in docBeforeSave is that correct? Because once the document is loaded these properties will have been added.

4. data-dot-path
Saving just stopped working in any locale once cursor.js is loaded. An the only thing I can see that may be causing issues is that the value of the data-dot-path attribute is changed from the original area name, to one containing localize.[local].[areaName], I am guessing this is why saving is not happening in the backend.

---

Apologies if any of these seems trivial, I am trying to work my way around 2.0. Again if you can help me out will save me tons of hours

Best,
Fotis

[fotisp]

Sorry to bump this, and I don’t know that the forum best practices are I am really stuck right now and really need to get a couple of projects out the door with this.

[boutell]

Hi Fotis, while I don’t have time to do a deep dive on it this week, we have an enterprise client who is expressing interest in seeing this functionality completed for 2.x, and possibly internationalization of URLs as well. There’s a good chance of that work taking place in the next month or so.

[fotisp]

Hello Tom,

I got a working version here

fotisp/existanze-docs-localized

Again its the old code, copy & pasted in the right places.

However I do have some apostrophe questions I would like to ask, to make the implementation better.

1. How can I access module properties in the ./lib/cursor.js file to avoid this

2. I had to add this helper in a middleware, because req is not available in afterConstruct call.

3. I understand the diff you are doing here, however there are some situations which I haven’t pin-pointed when, where the JSON.stringify is throwing a circular recursion error. So there needs to be another way to compare these before / after objects

4. For some reason which I can’t debug I have to do to lines index.js 128 to 132 because req.locale is always set to en. The same happens here ./lib/cursor.js lines 35 to 46.

5. This line here where _dotPath is carried over, causes problems in the frontend so I have to reset it. I don’t know if this is the right way.

This is a general observation here, we need some logging framework for A2.

I hope you can provide some feedback.

Best,
FP

[fotisp]

Hello All,

Apparently there are others who are using the module already, so we added a version which will be more closely maintained as we work through all the kinks.

existanze/existanze-docs-localized

FP

[fotisp]

Hello Again,

Sorry to wake this up again, but I am having a huge issue with middleware order and setting req.locale.

Specifically: the load method in apostrophe-pieces-widgets/index.js is being called before the middleware which sets the req.locale based on the URL pattern [url]/[locale]/[remainder].

This causes issues since it is always sets the locale to en, which in turn means that the logic in the “localized” filter will always load the English language.

Any ideas or suggestion?

FP

[boutell]

Hi Fotis,

There is a new effort on our end which does address some of this and will also eventually address localization, although it’s possible that more than one solution should exist to localization depending on whether there is also a need for a workflow process.

On behalf of an enterprise client we are currently building both workflow and localization features for 2.x, in such a way that localization is a natural extension of the work on workflow. It is a very different technical approach with a lot of benefits.

And of course we ran into exactly the issue you are describing. So there will soon be a feature allowing you to schedule your middleware to be added “before” specific other modules’ middleware.

I would expect that to land in the next week or two. It almost landed this week but there are just a few too many changes related to accommodating the needs of the new workflow module and we’d rather land them as a group.

See the workflow-accommodations-1 branch of Apostrophe. Also see the 2.x branch of the apostrophe-workflow module (very much a work in progress, lots of things broken or simply not even there yet, before you ask).

[ksinton]

@boutell Hi Tom,

We have a fairly immediate need for localization too. Do you have a ETA for when a module supporting that functionality, or a upgrade to apostrophe with that functionality will be available?

[ehibes]

Hi,
I made a module inspired by original apostrophe-localization. If you are interested, it’s here : https://github.com/ehibes/studiowaaz-localization

[bobclewell]

Hi @ehibes. Have you taken a look at our apostrophe-workflow module?

[ehibes]

https://github.com/ehibes/studiowaaz-localization/issues/1