SceneOntologyKnowledgeGraphEmbedding

Applies knowledge graph embedding techniques to the visual genome scene ontology data set for scene semantic understanding.

Generate Dataset(generate_dataset.py)

Generate training, test, and validation datasets from Visual Genome scene ontology dataset for knowledge graph embedding for scene semantic understanding.

Train Scene Ontology Knowledge Graph Embedding (ad_kge_train.py)

Run designated model on autonomous driving scene visual genome ontology datasets.

Run All Models (run_all_models.sh)

Generate knowledge graph from a valid dataset, train TransE, TransH, Rescal, HoLE knowledge graph embedding algorithms, and produce plots and results.

Dependencies and Requirements

Python
Pip
Github
Pytorch
Anaconda
pykg2vec

Follow instructions to install these repos or use the install script

Install Instructions that are also a bash script


echo "Starting setup."
mkdir ~/Desktop/ad_kge
cd ~/Desktop/ad_kge

read "Ctrl + C if you don't want to download py2kg env. Press [Enter] if you would like to continue."

conda create --name pykg2vec python=3.6
conda activate pykg2vec
conda install pytorch torchvision cudatoolkit=10.1 -c pytorch
conda install pytorch torchvision cpuonly -c pytorch
git clone https://github.com/Sujit-O/pykg2vec.git

read "Ctrl + C if you don't want to create dirs. Press [Enter] if you would like to continue."

cd pykg2vec
mkdir scene_data_1
mkdir scene_data_2
mkdir scene_data_3
touch scene_data_1/ad_scene_relationships-test.txt
touch scene_data_1/ad_scene_relationships-train.txt
touch scene_data_1/ad_scene_relationships-valid.txt
touch scene_data_2/ad_scene_relationships-test.txt
touch scene_data_2/ad_scene_relationships-train.txt
touch scene_data_2/ad_scene_relationships-valid.txt
touch scene_data_3/ad_scene_relationships-test.txt
touch scene_data_3/ad_scene_relationships-train.txt
touch scene_data_3/ad_scene_relationships-valid.txt
python setup.py install

echo "test install using: train TransE using benchmark dataset fb15k"
echo "Download the 3 datasets from: https://visualgenome.org/api/v0/api_home.html"
echo "Place them in a folder called: scenes"
echo "Remember: x for 1, 2, 3"
$ echo "Rename them: relationships_x.json"
$ echo "Structure should be as follows: 
/ad_kge/
    /examples/
        ad_kge_train.py
        run_all_models.sh
        train.py
        tune_model.py
        inference.py
    /pykg2vec/
        /scene_data_x/
            /ad_scene_relationships-test.txt
            /ad_scene_relationships-train.txt
            /ad_scene_relationships-valid.txt
            also results after training
    /scenes/
        /relationships_x.json
    generate_dataset.py"
    
read "Press [Enter] if you would like to continue."
echo "Then cd to ad_kge and run run_all_models.sh"
echo "This will build all the scene data and train four different knowledge graph embedding models on two sets of autonomous driving data."

aalbert-dev / sceneontologyknowledgegraphembedding Goto Github PK