ljvmiranda921 / gym-lattice Goto Github PK

I think it's better for our observation space to be a grid rather than a chain:

• Create a matrix of shape (m, m) matrix where m = 2 * len(seq) + 1
• Get the midpoint of that grid and treat that as origin. That's why you want to have odd number of cells
• The grid should have values in {-1, 0, 1} for polar, unassigned, and hydrophobic separately

I feel like the trapped check would be better to put after a move has been inserted into self.state. Otherwise there's no way to tell from the top-level program that we're trapped.

Right now I think we force the algorithm to take an extra step before it realizes "Hey, I was trapped after the previous move". It would be better to end the simulation immediately. Again, I'm willing to take this up if that's ok with you.

It would really be nice to add some automated tests and continuous integration for our base environment so that it's easier to extend in the future. I recommend using the pytest library. It's probably a good learning experience to do away with unittest for a while.

What if instead of reinventing everything, we just use some classes already implemented in gym?
Here's a good example from the docs itself. Just study the API and see how things will work out from there.

This won’t really break a lot of things, and it eliminates the cost of creating a grid from a sequence length. The problem is that we tend to create unnecessarily large grids (well to further accommodate the edge case worst case of a one-directional chain), and we need to build a rid of just the right size

You shouldn’t include H’s that are consecutive to one another inside a lattice.

There is a possibility, given long sequences, that your structure will be “trapped” inside its own structure. This means that there are no more available moves and you still have polymers to be placed. How will you resolve that?

My idea is to forcefully break the loop. Give a large penalty when this happens.

When you render (just using the sample random agent), up/down seem to be swapped:

  (Left)
***********
***********
***********
***********
***********
****HH*****
***********
***********
***********
***********
***********
  (Right)
***********
***********
***********
***********
***********
****HH*****
***********
***********
***********
***********
***********
  (Down)
***********
***********
***********
***********
****P******
****HH*****
***********
***********
***********
***********
***********

I'm guessing this is related to np.flipud? I'd be happy to open a PR fixing this if that's fine by you.

Hi, awesome work on the gym environment! I am wondering is the training specific for each HP sequence?

Is the current design such that we have to start over the training, for example, train a new q-table for each new sequence?