blackfoundrycom / django-robo-cjk Goto Github PK

For example this commit:

• notofonts/noto-cjk-varco@be34976

It appears these get reinstated during the nightly full export, but this makes the data close to useless during the day: a manually triggered font build will fail.

Part of googlefonts/fontra#1050

The data section of the response should include a groups key and a permissions key. Their values are lists.

The readme has the REST api URLs but doesn't explain what this project is about :)

When we create a glyph with

atomic_element_create
deep_component_create
character_glyph_create

it automatically lock the glyph for the designer.
But then when the designer want to open the glyph to edit it, RoboCJK relock it again. So RoboCJK see twice a lock and avoid modification.s

I think we should remove the automatic lock of these three creation function. But does this will affect you @justvanrossum or not?

I think this section:

https://github.com/BlackFoundryCom/django-robo-cjk/blob/bc0b2374612379c8ac1f022de8d30a63a4b4dd92/robocjk/api/client.py#L112-L117

Should be followed by something like:

elif response.status_code != 200:
    raise HTTPError(f"HTTP error: {response.status_code}")

The standard django email logger sends an email for each logger.error call, this is good but could end up in sending hundreds of emails per hour in some scenarios, especially when there is an error during the export.

To avoid receiving multiple emails for the same kind of error in a short time interval we need to add a custom logger.

This would be an extension to #14, and depends on #11 to be resolved.

It's been apparent for a while that the user name added to the commit message is not precise: it appears to often add more names than were actually involved in the change. With the incremental export, this becomes even clearen. In this case it even lists a person who hasn't worked on the project for a while:
https://github.com/BlackFoundryCom/gs-cjk-rcjk/commit/e6278c3c55446a41d8325e4b698aba3bdda59a12

If I try to create a layer name that contains a non-BMP char (unicode > U+FFFF, say "𠃓", U+200D3, or "👀", U+1F440), the server throws an error:

HTTPError('500 Internal Server Error - (1267, "Illegal mix of collations (utf8mb4_general_ci,IMPLICIT) and (utf8mb3_general_ci,COERCIBLE) for operation \'=\'")')

The same happens when using such a char in a glyph name.

As far as I can see, this is coming from MySQL.

Not high priority, and, like the long layers names, I could work around this in Fontra, but it's probably good to know.

• #13

• #14

In the font_update() function, the if xxx: conditionals will evaluate to false for empty values, and will then not update the field. This makes it impossible to clear any of those fields.

I can't find the definition for get_dict(), so I don't know what it returns for a missing argument. There should be a distinction between "argument not given" and "argument is empty".

We definitely need may want to raise the limit, but I'm thinking whether we can come up with a scheme that lifts the limitation completely.

We could shorten long layer names for storage by truncating them and appending a hash. This is easy if we only need to map from "long layer name" to "shortened layer name".

Alternatively, we can work around this in the rcjk Fontra backend, and do the mapping there.

Traceback (most recent call last):
  File "/root/robocjk/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/root/robocjk/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/root/robocjk/lib/python3.8/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
    return callback(request, *args, **kwargs)
  File "/root/robocjk/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/root/robocjk/lib/python3.8/site-packages/django/views/decorators/http.py", line 40, in inner
    return func(request, *args, **kwargs)
  File "/root/robocjk/src/robocjk/api/decorators.py", line 42, in wrapper
    raise internal_error
  File "/root/robocjk/src/robocjk/api/decorators.py", line 39, in wrapper
    response = view_func(request, *args, **kwargs)
  File "/root/robocjk/src/robocjk/api/decorators.py", line 90, in wrapper
    return view_func(request, *args, **kwargs)
  File "/root/robocjk/src/robocjk/api/decorators.py", line 153, in wrapper
    return view_func(request, *args, **kwargs)
  File "/root/robocjk/src/robocjk/api/decorators.py", line 436, in inner
    return view_func(request, *args, **kwargs)
  File "/root/robocjk/src/robocjk/api/decorators.py", line 77, in inner
    return view_func(request, *args, **kwargs)
  File "/root/robocjk/src/robocjk/api/decorators.py", line 202, in wrapper
    return view_func(request, *args, **kwargs)
  File "/root/robocjk/src/robocjk/api/views.py", line 616, in character_glyph_update
    character_glyph.save_by(user)
  File "/root/robocjk/src/robocjk/abstract_models/core/timestamp.py", line 58, in save_by
    self.save()
  File "/root/robocjk/src/robocjk/models.py", line 997, in save
    self._update_status(glif_data)
  File "/root/robocjk/src/robocjk/models.py", line 943, in _update_status
    if init_val == 4 and val < init_val:

Exception Type: TypeError at /api/character-glyph/update/
Exception Value: '<' not supported between instances of 'NoneType' and 'int'

It is val which is None here.

Expected 81, found 71 atomic elements layers .glif files on file-system.

https://robocjk.black-foundry.com/admin/robocjk/atomicelementlayer/?font=24

It would be very useful for Fontra to be able to know the modified time stamp for glyph data, and I imagine the character_glyph_get, deep_component_get and atomic_element_get calls could pack this into the response object.

These fields should only be updated when actual data changes.

The current behavior gives me false a positive on "updated_since" when a glyph is only opened in RoboCJK, and not even edited.

I realized that simply having the list of glyphs that have been modified since a certain time stamp isn't quite enough for my use case.

I can explain the use case in more detail if you want.

Relates to #13. With a call like that, Fontra can poll, say once per minute, so it can let clients know glyphs have been changed in the meantime.

It would be good for me if this call would bundle all changed CG's, DC's and AE's into a single response (a list per type).

The response should only contain glyph IDs, not glyph data. Fontra will request fresh data if needed, but it is expected that this is often a subset of the set of actual changes.

I imagine the call to be defined something like this:

    def glyphs_modified_since_list(self, font_uid, timestamp):
        ....

And the return value to be something like this:

{"ae": [1, 2, 3], "dc": [5, 6, 7], "cg": [9, 10, 12]}

The objective is to avoid invalid errors like:

Expected 0, found 35 atomic elements .glif files on file-system.

To avoid that these kind of invalid exceptions will be raised 2 changes are necessary:

• Raise exception only if the expected glif files count > 0
• Change delta tolerance of expected glif files to 0 <= n <= 50

This is about

• character_glyph_lock / character_glyph_unlock
• deep_component_lock / deep_component_unlock
• atomic_element_lock / atomic_element_unlock

I do like the fact that unlock also returns the updated_at and layers_updated_at fields, but the glif data is redunant for my use case, as I will only lock after I've long downloaded the relevant glyph data.

Glyphs that are deleted from the DB only get deleted once a day, instead of immediately upon export.

This is currently blocking a release of GS CJK.

Currently, if a user owns the lock for a glyph, the same user can ask for the lock again, even from another client. This also implies: user A on client B can unlock the lock owned by user A on client A (as the client ID is not taking into account).

A practical scenario where this will cause problems (or at least confusion):

• User A opens a glyph in RoboCJK.
• RoboCJK locks the glyph on behalf of user A
• User A opens a glyph in Fontra
• User A edits the glyph in Fontra
• Since user A owns the lock, Fontra will succeed with:
  • getting the lock
  • updating the glyph
  • unlocking the glyph
• Now the glyph is unlocked, even for RoboCJK, but RoboCJK doesn't know that: it still thinks it holds the lock
• Next up: confusion and/or errors and/or lost edits

This is expected to be a rare thing, given our path forward with Fontra, and we need to decide carefully whether it is worth fixing at this point.

(Note that the plan is for Fontra to have its own locking layer, which will will indirectly prevent this problem from being one if all edits are done via Fontra.)

A possible solution for this is:

1. Disallow locking an already locked glyph even by the same user
2. The response to a succeeded lock call should contain an “unlock key” (a random token made by the server)
3. Unlocking requires passing the unlock key

A glyph can have multiple code points, so that any of those code points will be mapped to that glyph in the final font.

The data currently returned by glif_list only contains the first of such code points in the response, in the unicode_hex field.

Fontra needs to be able to construct a complete cmap from the glif_list output, so it needs all the code points associated with each glyph.

(In .glif data, the <unicode> element can appear multiple times.)

(An example of a multiple-encoded glyph is uni313B in GS CJK Hangul.)

As discussed in f2f, this may be an option to help implement authentications surviving a Fontra server restart.

Or: how does a client know what timestamp to use when requested updated_since glyph lists?

Should responses perhaps contain the current time (timestamp of the response), according to the server?

Or should there be an additional api for "current time on the server"?

It is currently not possible to create a glyph with the name a when a glyph A already exists.

Error:

400 Bad Request - Character Glyph with font_uid='263d7da5-16c0-4a2f-b5bb-763a1b082d6b' and name='a' already exists.

('a' does not in fact exist, but 'A' does.)

Some files have not been included in the incremental export, then after having modified them they have been included in the next export.

The cause of this issue is probably related to the glif creation date VS the project/font export start date.

Using a Session, api calls take up to half the time compared to without using a Session. Probably mostly because of the reuse of the connection.

--- robocjk/api/client.py	2021-12-09 21:37:40.000000000 +0100
+++ /Users/just/code/git/fontra/vault/client.py	2021-12-09 21:34:18.000000000 +0100
@@ -56,6 +56,8 @@
         if not password:
             raise ValueError('Invalid password: {}'.format(password))
 
+        self.session = requests.Session()
+
         # strip last slash in case
         if host.endswith('/'):
             host = host[:-1]
@@ -107,7 +109,7 @@
             # 'verify': self._host.startswith('https://'),
         }
         # send post request
-        response = requests.post(url, **options)
+        response = self.session.post(url, **options)
         if response.status_code == 401:
             # unauthorized - request a new auth token
             self.auth_token()