Comments (14)
Hi @poojitharamachandra , thank you for your interest. The GRIP works in 3D space. If you look at the input dimensions, it does not have a specific design to only process 2D data. Thus, we can simply replace 2D space data with 3D space data before feeding them into the model. We did not try the model on a 3D space trajectory dataset, but we tried the model on the Human3.6 for human pose (3D coordinates) prediction and achieved better results than state-of-the-art solutions.
from grip.
thanks for the info. Could you please share your work on Human3.6? if possible
I have couple of questions:
1.what is the significance of max_hop?
2.could you explain in brief the convolution and einsum operations of graph convolution? especially what each dimension specify?
3. I am trying to apply your concept on the system of particle trajectories confined in a definite space with periodic boundary conditions. i.e, if the particle flies out from one side of an imaginary box, it returns on the other side of the box. Do you think this is feasible with your model?(maybe with some modifications)
from grip.
-
The "mat_hop" is passed into Class Graph() in layers/graph.py. It is used in GRIP/layers/graph.py [line 21], np.linalg.matrix_power(A, d). In Graph Theory, adjacency matrix powers (A^hop_step) gives the number of walks of length hop_step from node i to node j. Here, we use it to figure out whether two nodes are connected within "hop_step" steps.
-
In forward() function in layer/graph_operation_layer.py [line16-34], adjacency matrix A has a size of (n, k, v, w), where "n" is the batch_size, "k" is the number of matrice (we have multiple matrice, connection to the node itself, connection to other nodes, etc.), "v" and "w" corresponding nodes. Thus, in this function, we reshape the input feature "x" to match the dimensions with A, and then do matrix multiplication. The einsum allows us to do matrix multiplication along specific dimensions. (Please refer to the official document for more details.)
-
Yes. GRIP should be able to handle the case you described. You can use the same network architecture, but the dataloader should be modified to handle your own data. Meanwhile, please refer to the "TABLE III: Changes in performance (in term of WSADE) while adjusting the model" in our GRIP++ paper (https://arxiv.org/pdf/1907.07792.pdf).
In addition, I reported the results on the Human3.6M in this thesis (https://core.ac.uk/download/pdf/304204351.pdf) since Page 61.
from grip.
Thanks for the detailed information :)
I see that you are using 6 history frames to predict for 6 future frames. But the neighbor matrix does not contain any time dimension. could you please let me know what is the reason for this.(I understand that you are using only the neighbor matrix of the start frame)
Also I don't see usage of Graph.get_adjacency anywhere (may be I have mistaken?)
from grip.
@poojitharamachandra , we only take objects that appear at the last observed time step into account (data_process.py line48) because objects that do not appear at this time step have a high probability do not appear in the future. In addition, objects may have occlusion (disappear) on some frames and then come back during our observation. If we generate an adjacency matrix for every single frame, the relationship between objects may be changing quickly (not consistent, it has a negative impact on the feature extraction: missing some features on/before those occlusion frames, because features are multiplied with the adjacency matrix). Thus, we just choose the last observed frame to calculate the adjacency matrix.
The Graph.get_adjacency is called in xin_feeder_baidu.py (line 83).
from grip.
@xincoder thank you very much
How do You ensure that the predicted vehicle trajectories do not overlap? (I have this problem in my usecase)
from grip.
@xincoder thank you very much
How do You ensure that the predicted vehicle trajectories do not overlap? (I have this problem in my usecase)
@poojitharamachandra It is a very good question. In our work, we do not add any specific design to avoid predicting overlapped trajectories. In real life, should we force the model not to predict overlapped trajectories? (If there are predicted overlapped trajectories, does it mean two objects may collide in the future?)
from grip.
I am trying to use your paper to as inspiration for a different (physics) problem . in my use case it is important that the molecules don't collide with each other. (also I am predicting the positions, not velocity)
from grip.
@poojitharamachandra Thank you for adapting our work in solving other problems. Back to your question, we do not use any specific design to avoid colliding. There are two simple ways that you may want to try: (1) during training, you can add a big loss if there is any collision, (2) adding some post-processing to modify the prediction if there is any predicted collision. (Hope these two simple ideas inspire you to come up with more efficient and effective schemes.;-))
from grip.
thanks!
from grip.
Hi,
- Could you please tell me how are u augmenting the data?
- Is the representation learned by graph convolution independent of the absolute co-ordinates (x,y) gives as input?
- You mention that the graph convolution model learns 'good representation'. Is there any analysis done on what kind of representation it learns?
Thanks
from grip.
Hi @poojitharamachandra , thank you for your interesting in our work.
- Please refer to the GRIP/xin_feeder_baidu.py (function getitem, line59) for the details of our data augmentation. Durining training, we mainly do randomly rotation augmentation (line65-81).
- What do you mean by saying "independent of the absolute"? As we descripted in our paper GRIP++ (Section IV. A. 1 Input representation), "we calculate velocities before feeding the data into our model".
- There was no special analysis (neither tried to explain what representation the model learned) about the representations. However, the better final predictions prove that the model learned a good representation.
Hope this answers your questions.
Thanks.
from grip.
thanks.
could you please point to the code where the velocities are calculated. (I am guessing the augmentation is done before calculating the velocities)
from grip.
@poojitharamachandra The velocities are calculated in main.py [line 99]. Thanks.
from grip.
Related Issues (20)
- RuntimeError: cuDNN error HOT 3
- Visualize results
- The evaluation results are different HOT 4
- About the test results HOT 8
- Argoverse数据集的训练与测试问题 HOT 2
- Is grip++using heading? HOT 4
- velocity input HOT 2
- In data_process.py line 15 HOT 2
- input preprocessing model HOT 4
- Integrated with HD map (vector map)
- xin_feeder_baidu.py file HOT 2
- How can I install layers.graph_operation_layer in graph_conv_block.py
- model文件中出现了keyerror
- 论文中Fig.5中,有车道线信息,请问这个车道线信息是从apollo数据集中获取的吗? HOT 1
- GRIP code
- GRIP++ on NGSIM HOT 14
- About dataset HOT 1
- GRIP++ on NGSIM
- the question of Urban Traffic Scene(Sample)
- 有关行进方向
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grip.