New field additions could easily be handled through "Issue" tickets here on GH, where their utility can be discussed before they're added to the system.

from adventurelookup.

I'm going to take a stab at "Settings" as a static model in a branch on my side, just to gauge the complexity of moving a single, straightforward data model. Planning on linking it via a new column on the Adventures table, setting_id or something like that. I'll conform to doctrine conventions where appropriate

from adventurelookup.

Also, getting rid of user-defined fields means we can greatly simplify the model representation and make it easier to access model fields, looking ahead to an API, for example. It will also get rid of a lot of looping over adventure.info.

from adventurelookup.

I believe this was core to Matt's original idea.

I think the mistake is having fields be global, instead of using a JSON blob per adventure to store arbitrary user-entered data.

Essentially, you should have:

1. Global, hardened fields (things we want in the data model)
2. Global, optional fields (things that can just be dropped into the JSON)
3. Per Adventure fields (JSON per Adventure)

Fields from group 1 are the only ones revealed in the site filters.
Fields from group 1 and 2 are available in the edit/create screens.

Users can created fields from group 3, per adventure.
Moderators can create fields from group 2.
Developers are required to add fields in Group 1.

This would contain the silly field problem to a per-adventure silo, minimizing system damage while maintaining Matt's original vision.

This isn't drastically different from what's currently there.

from adventurelookup.

The ability for editors/curators to programatically add new storage areas was something Matt thought was neat, but not part of the original requirements of the AL. I don't know if Matt ever came out and specifically says it, but from what I recall I got the feeling that the need to add or remove fields was born out of the experience that its unlikely we're going to get everything right the first time around. Whether that is done via changes to the models in the source code or someone going in and doing it in the interface probably doesn't matter much (except for where one or the other has meaningful advantages/disadvantages).

Having the "adventures" data structure be dynamic based on a row lookup in another table, unless it is likely to change frequently (I don't think so?) or we need the flexibility of handling different sets of fields for different types of adventures (we don't, but even if we did there are still better ways of handling this), has negative impacts on performance and programming costs well above the meager benefits its offers.

Just my opinion on the matter. I think its worth considering, I don't think we could move forward on this without Matt understanding the scope and giving his opinion on the matter.

from adventurelookup.

Yeah, @MattColville will need to confirm his vision. I can easily design a UX to solve either case.

from adventurelookup.

I'm a huge opponent of the "just add everything, what's the downside?" philosophy. I don't think anyone here is proposing that, but it informs a lot of my decision making.

If the site is overburdened with options, especially options that aren't immediately, obviously, useful, people will get overwhelmed when they land here and give up.

In that spirit I'm opposed to letting "everyone" add fields. I've already seen a lot of people wanting to add new fields and in some cases adding them. Their hearts are in the right place, but I think some of their choices are ill-advised. Or, at least, I disagree with their utility function.

I think we should restrict adding new fields to administrators. It would be nice if curators could add new fields, but they aren't visible to customers unless an administrator approves them.

But really, I think we're close to just not needing more fields. We could just lock the whole thing, and I think the site would chug along swimmingly.

from adventurelookup.

That makes things way way easier in so very many ways (yay!).

Would you be OK with developers alone being able to add fields? That'd remove the need for an administrator field UI, speed up development and make it much easier to handle … well just about everything.

from adventurelookup.

There are other benefits to moving towards static models. Certain data can be "special." For example, a "Monster" need not JUST have a name. It can also have a "Type" so that I can find all adventures set in the "Mountains" with "Giants" and I'll get all the adventures with giants, but maybe I'll also get a few troll-specific adventures I hadn't originally even thought about, but would work well.

from adventurelookup.

Or the "Setting" can be a drop down list that users chose from, instead of a free-type text.

from adventurelookup.

The move to static models would open up a good number of awesome options.

We could, for instance, import all known MM monsters (names only) from the numerous XML sources out there, speeding up entry and normalizing their names.

from adventurelookup.

I'm going to use the setting example, because I think it's a good one.

I have two, contradictory thoughts and I'm not sure which should win. What do you folks think?

1: We really shouldn't let users add settings. Because (if they haven't already) folks will add their own settings that not only has no one ever heard of, they actually aren't available. Someone publishes an adventure on the DMs Guild, very good! Very nice! Happy to have you aboard. But now they want to add the homebrew, unavailable setting for it to AL.com and now we've got settings in the dropdown no one's ever heard of.

I would like to see a dropdown, but I'd like that dropdown to have like a break in it. List the three to five most popular settings, break, huge list of alphabetized settings. Silly to make someone scroll down to F for Forgotten Realms when we all know how popular it is. Like putting United States at the top of a country field for shipping.

2: Do we even need a dropdown? Setting is not, I submit, one of those fields folks want to monkey with other than picking one. Folks can just type "Forgotten Realms" and filter. Bob's your uncle.

I would honestly love to see some excellent, well-supported, but obscure settings get some love because of this site. Mystara, Wilderlands of High Fantasy. That's the benefit of the dropdown. Folks will get exposed to stuff they might not otherwise have heard of. But that's....that's not a good reason to do this. :D

from adventurelookup.

Adding a "featured" list at the top is totally doable and isn't unheard of.

As for the other points you made, both 1 and 2 are handled in dropdown functionality that allows you to combine functionality from both typing and selecting from a dropdown: https://jedwatson.github.io/react-select/

That's just one example built using react. There are a ton of them out there.

from adventurelookup.

I tend towards 1, if only because I believe consistent, constrained data is the best possible user experience.

If, and I believe this will be true, there's a large enough pool of contributors, it should be possible to have a well-stocked list of all "notable" settings, which would be commonly searched for by users.

These should be the settings which appear in the search filters.

However, for Adventure Submission, we can use an autocomplete (see UX for a mockup in the Title field) to make entry fast and consistent. I'd personally continue to allow users to enter arbitrary settings, since that could act as a flag for settings which are more popular than administrators might realize, and avoid dumping every non-already-added-setting under a "Homebrew" label.

So a constrained set in the filters, but the allowance for entry of arbitrary settings by submitters.

from adventurelookup.

I agree with the consistent, constrained data. Let's make it a dropdown for lookup and an autofilled field for submission.

from adventurelookup.

I totally agree to the move to static models instead of the current approach. It will make the templates and forms much easier to develop.
The only reason, really, I made the fields dynamic, was that I feared Matt saying: "Ahhh, well, your prototype looks nice, but it needs fields A, B, C and shouldn't have field X and Y. Sorry - won't use it!". I wanted to make sure I didn't have to put a lot of work into changing the fields until he liked them.
Now that it's being used, and the work will not be wasted, the dynamic fields cause more problems than good. I have the day off (except for a D&D game 💯), so I'll start moving things into a static model.
This will also fix the bug which prevents the site from being updated right now.

from adventurelookup.

@jsaugustyn @etchalon I have been thinking about moving to a static data model for a fair bit now, but I haven't come up with THE way to do it.
There is two parts to consider:

1. The representation of an adventure inside Elasticsearch
2. The representation of an adventure inside the MySQL database

Elasticsearch representation

The first one is pretty straightforward. Elasticsearch doesn't need to care about who added an adventure, who proposes an addition to an adventure, and so on. Elasticsearch only ever has the latest and fully approved adventure version in its search index. It looks like this:

+---------------------------------+
|                                 |
| Elasticsearch AdventureDocument |
|                                 |
| title                           |
| monsters[]                      |
| villains[]                      |
| environment[]                   |
| startingLevel[]                 |
| setting[]                       |
| author[]                        |
| ...                             |
|                                 |
+---------------------------------+

*currently, also unapproved fields are part of the search index, because we don't really have structures to approve adventures quite yet. Also, the fields are named info_x because they are still dynamic. This is improved by #29.

MySQL representation

Current state

This is a bit more complicated. Not only do we need to save multiple versions of fields, we also need to save who proposes to add / edit / delete which field and whether or not a field is accepted. Changes need to be logged to be reverted, etc. Currently, this is done using the following schema.

+-----------+      +------------+        +---------+
|           |      |            |        |         |
| Adventure |      | TagContent |        | TagName |
|           |      |            |        |         |
| title     |1    n| content    |n      1| title   |
| slug      +------+ version    +--------+ desc    |
|           |      | approved   |        | example |
|           |      | created_by |        |         |
|           |      |            |        |         |
+-----------+      +------------+        +---------+

*please note that in most places, 'Tag' has been renamed to 'Field' already.

All edits/deletes of TagContent content are automatically saved into a separate ext_log_entries table. This table isn't used anywhere in the frontend yet, but could be used for #28 for example.
Here's an example from my local installation:

+-------+--------+---------------------+-----------+-----------------------------+---------+----------------------------------------------------+----------+
| id    | action | logged_at           | object_id | object_class                | version | data                                               | username |
+-------+--------+---------------------+-----------+-----------------------------+---------+----------------------------------------------------+----------+
| 39576 | create | 2017-07-18 08:04:56 | 39535     | AppBundle\Entity\TagContent |       1 | a:1:{s:7:"content";s:22:"Mrs. Esperanza Abshire";} | NULL     |
| 39577 | create | 2017-07-18 08:04:56 | 39536     | AppBundle\Entity\TagContent |       1 | a:1:{s:7:"content";s:22:"Dr. Izaiah Witting DDS";} | NULL     |
| 39578 | create | 2017-07-18 08:04:56 | 39537     | AppBundle\Entity\TagContent |       1 | a:1:{s:7:"content";s:11:"5th Edition";}            | NULL     |
| 39579 | create | 2017-07-18 08:04:56 | 39538     | AppBundle\Entity\TagContent |       1 | a:1:{s:7:"content";s:17:"Necromancer Games";}      | NULL     |
| 39580 | create | 2017-07-18 08:04:56 | 39539     | AppBundle\Entity\TagContent |       1 | a:1:{s:7:"content";s:8:"Greyhawk";}                | NULL     |
+-------+--------+---------------------+-----------+-----------------------------+---------+----------------------------------------------------+----------+

Future state

If we now move this into a static model with all-predefined fields, I see two extremes:

Completely denormalized

All fields, even arrays, are part of the Adventure table. It would look almost identical to the AdventureDocument above.

Split over multiple tables

We'd have monsters, authors, items, etc. tables. This appears to be way more complicated.

Do you guys have thoughts on the matter?

from adventurelookup.

I prefer a normalized approach, there are a couple of more complex architectural approaches to maintaining history that are good solutions to this problem and a few others that are simpler, but still acceptable choices. It's not an unsolved problem in the data science community, consequently I don't think it's worth the losses associated with a denormalized model.

from adventurelookup.

from adventurelookup.

I've not used Doctrine a ton, but if its anything like other ORMs, if the data is stored in an external table you need an "Entity"/Model for the ORM to work properly. There are some classic, iconic items with complex naming. I don't think we'll do anything special with them, but putting them into a table could be one way to keep misspellings to a minimum (items would work as a lookup, similar to monsters, or settings, albeit with a different UX since its expected that you'll select many items per adventure, rather than just a couple).

from adventurelookup.

This has been tacked, thanks largely to cmfcmf!

from adventurelookup.