Allow `value::Object` to hold complex objects about liquid-rust HOT 10 CLOSED

I've been back and forth on doing something like this.

One question is how heavy of what kind of site is cobalt targeting?

For ideas within our current setup, I've been considering switching Liquid's stack to use Cow and then having the sources of information to be different layers in the stack rather than constantly merging them.

Another level of optimization is turning off prev/next and index pages on a collection by collection basis.

With that, I'm unsure what advantage a trait would be

from liquid-rust.

I guess I've conflated 2 things.

1. The ability to not load unused data. Shopify/liquid allows object properties to return Ruby Proc to this end. If my_object.toto returns a Proc, Liquid will call the proc and memoize (albeit badly) the returned value.

   Given the following context:

   {
     "collections" => fn() {
       load_all_collections().map(|collection| {
         { 
           "title" => collection.title(),
           "products" => fn() {
             load_products(collection).map(...)
           }
         }
       }
     }
   }
   

   The collections don't have to be loaded if the template doesn't access them. I understand that in the interest of a static site generator this may not have great value, but for dynamic sites, not loading unused data is useful.

2. The ability to have mutate objects from within blocks so that tags within the block's scope see the mutated objects. This isn't required, but would definitely be nice to have.

   With that, I'm unsure what advantage a trait would be

   I... I'm not sure it's the way to do things, my Rust toolkit isn't super great.

   I imagine it would allow us to do something like

   trait Object {
     fn get_val(&self, name: &str, context: &Context) -> Option<Box<Object>>
     // other functions currently in the `Value` impl
   }
   
   impl Object for Value {
     // basically current implementation
   }
   
   impl Object for LazyCollection {
     fn get_val(&self, name: &str, contexT: &Context) -> Option<Box<Object>> {
       match name {
         "products" => load_products()
         x => self.properties.get(x)
       }
     }
   }
   
   struct Context {
     stack: Vec<Box<Object>>
   }

from liquid-rust.

The ability to not load unused data.
I understand that in the interest of a static site generator this may not have great value, but for dynamic sites, not loading unused data is useful.

So you are interested in using liquid-rust outside of cobalt and instead in a dynamic website?

As you mentioned, for static site generators, whats the point of having data if you don't use it, so everything will eventually be loaded. The main issue from there is

• Reducing unnecessary copies which I mentioned I have ideas for
• Reduce peak memory usage. I think we'd be blocked on a concrete use case where peak memory is a problem so we know how best to resolve it.

The ability to have mutate objects from within blocks so that tags within the block's scope see the mutated objects. This isn't required, but would definitely be nice to have.

So right now, if you do an assignment over another variable, instead of overwriting it, we shadow it causing the lifetime of that mutation to be for the current scope. You are interested in having ways for us to actually overwrite the value?

You mention this isn't required but it'd be nice to have. What problems are you running into that it'd help solve and what are you instead doing to work around them?

With that, I'm unsure what advantage a trait would be

Sorry, my comment wasn't as much about the method (a trait would be the right approach) but use case being handled by the method. The original issue talks about changing the internal implementation with a specific example but I'm trying to understand the motivation. What is this buying the template writer or the person integrating liquid-rust into their project?

I find "whys" make it a lot easier to ensure the goals of the people are aligned (is liquid-rust or cobalt the appropriate tool), discover more about the requirements, and ensure the solution meets people's needs.

from liquid-rust.

So you are interested in using liquid-rust outside of cobalt and instead in a dynamic website?

Yes that's correct!

peak memory is a problem

To be fair, you could still render pages individually, so only the backing models would have to be in memory, which is what Jekyll does. In Ruby. Some sites have tens of thousand of articles. Cobalt probably can't do worse!

What is this buying the template writer or the person integrating liquid-rust into their project?

For me, it's only about not performing queries (SQL or otherwise) to load data the template doesn't use. I can always try to infer the types of data used, and only load that, but that would be tricky since Liquid is Turing-complete.

What problems are you running into that it'd help solve and what are you instead doing to work around them?

This is mostly to deal with a legacy tag that mutates the 'window' of a slice of data

{% in_groups_of 3, collection.products %}
  <row>
    {% for product in collection.products %}
      here `collection.products` has 3 products
    {% endfor %}
  </row>
{% endin_groups_of %}

I think it's possible to make light copies of the data in collection such that products is just a slice over the the existing products. Alternatively, I can reimplement the for tag.

from liquid-rust.

So you are interested in using liquid-rust outside of cobalt and instead in a dynamic website?

Yes that's correct!

Full disclosure: I'm hesitant about making a trait for value::Object. It'd have major performance implications. It'd be an invasive change.

It feels like it'd end up being complicated to actually solve your use case with. I'm assuming you'd end up lazily creating your child objects as you walk.the.tree and they'd need to duplicate the shared sub expressions for accessing the data in the database. Each one would also need an Rc<Database> since we probably wouldn't be passing lifetimes around for that.

Another possible solution to your problem is for context.get_val to call into a trait with a object.path and would return a value::Value.

The main user-facing downsides to this are

• Can't lazily load items for a large array
• You won't get hints as to what data type the template is expecting

An implementation question to resolve is how we identify which frame in the stack to do the lookup in. Maybe we do the lookup in every frame and the trait returns an Option.

from liquid-rust.

You bring some good points. I'll see if I can work around it, otherwise I'll try to come up with a solution. In the meantime I'm ok if we close this.

from liquid-rust.

Would the idea for a context.get_val Trait at least make things easier?

In the meantime I'm ok if we close this.

I hope you aren't taking my thoughts as discouraging solving this. I think its an interesting and useful case to handle.

For the maintainer role aspect, I'm still exploring what the owners want out of the project and am trying to be conservative.

from liquid-rust.

I'm assuming you'd end up lazily creating your child objects as you walk.the.tree and they'd need to duplicate the shared sub expressions for accessing the data in the database.

For my case, probably not. I would just guard the heavier objects, but if an access to it is made, the entire subtree would be loaded.

As for context.get_val, I'm not sure what that would look like, or how it would help.

For the maintainer role aspect, I'm still exploring what the owners want out of the project and am trying to be conservative.

Yeah that's totally understandable! You're already doing an amazing job at communicating you needs and understanding mine!

from liquid-rust.

For reference, #95 includes the get_val trait.

from liquid-rust.

#197 added a Globals trait. It is shallow. Hopefully this addresses the need in this issue.

from liquid-rust.