brefphp / dynamap Goto Github PK

I've spent a little bit of time looking at Dynamap and thinking about how to adapt it for supporting more advanced access patterns than a simple multi-table key/value store (eg; encapsulating an entire application's data store within a single table, as is the AWS recommendation)

Adding GSIs and LSIs to support m:m and o:m 'relationships' are probably the most daunting part of gettting to grips with DynamoDB, and something which I think we can support relatively easily.

However, I think this will necessitate some deep changes in the way Dynamap currently works. Documentation will also need to be clear about how best to use / implement the design patterns with Dynamap / DynamoDB, including defining your access patterns before you start creating tables where possible, and also recommending best practices such as using UUIDs for identifiers.

I was thinking that we could automate the generation of GSIs and LSIs by using something similar to Doctrine association mapping, using annotations (or perhaps PHP array-based config similar to what the current version of Dynamap uses).

To do this, we'll probably need to create a simple schema generation tool and compare it against the table definition. The knock-on of this is that we'll also need to be able to create migrations of some sort (or maybe just a schema update tool).

At the moment I think the design constraints mean we'd probably need to define one or more 'root' entities (eg Article), and allow relationship mapping between those and linked entities by creating a GSI or LSI depending on if the relationship is m:m or o:m. However, I'm not 100% certain on this. I need to do more reading / experimenting / talking to my rubber duck to see if it's actually the case.

I propose that we alter the hierarchy of the mapping to move the table element higher, in order to reinforce the notion that multiple entities can live inside the same wide-column table:

eg, to have:

$mapping = [
    'tables' => [
        'name' => 'articles',
        'mappings' => [
            Article::class => [
                'keys' => [
                    'id',
                ],
            ],
            Author::class => [
                'keys' => [
                    'id',
                ]
            ]
        ]
    ]

rather than:

$mapping = [
    Article::class => [
        'table' => 'articles',
        'keys' => [
            'id',
        ],
    ],
   Author::class => [
        'table' => 'articles',
        'keys' => [
            'id',
        ],
    ],

Adding relationships in (assuming a PHP array config rather than annotations at this moment in time), we could end up with something like this:

$config = [
    'tables' => [
        'name' => 'articles',
        'config' => [
// Set AWS table options, eg: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ProvisionedThroughput.html
            'ProvisionedThroughput' => [
                'ReadCapacityUnits' => ...,
                'WriteCapacityUnits' => ...,
                ],
        ],
        'mappings' => [
            Article::class => [
                'keys' => [
                    'id',
                ],
                'relationships' => [
                     Author::class => 'manyToOne', // inverse of 'oneToMany' on Author below
                     Tag::class => 'manyToMany' // will create a GSI
                ],
            ],
            Author::class => [
                'keys' => [
                    'id',
                ]
                'relationships' => [
                     Article::class => 'oneToMany', // will create an LSI
                ],
            ],
            Tag::class => // ...
        ]
    ]

Note: to support multiple entities in the same table, PK & SK fields will probably be need to be prepended with the entity name, eg Article-1319f90c-d8b2-46d9-ac2b-0255eee97374

I was planning on looking at roave/better-reflection to reconstitute the objects after fetching from DynamoDB & to inspect properties for mapping, but at the moment I'm not sure if it's necessary or not.

Edit: for the time-being I think I'll stick to PHP's own reflection API until I need better-reflection.

For reference, these videos talk about modelling relationships inside a single table with DynamoDB (both are well worth watching):
https://www.youtube.com/watch?v=ziqm6q-JsGQ
https://www.youtube.com/watch?v=HaEPXoXVf2k

WDYT?