Giter VIP home page Giter VIP logo

gnn-pressure-estimation's Introduction

GATRes

GATRes architecture Graph Attention NeTwork with Residual connections

This is the official repository for the paper: "Graph Neural Networks for Pressure Estimation in Water Distribution Systems".

UPDATE: (21/04/2024)

Some changes in default parameters have propagated you to a minimal example on C-town.

First, you can create a dataset based on the default config using the below command:

python scenegenv7.py --executors 2 --batch_size 50 --config "configs/v7.1/ctown_7v1__EPYNET_config.ini

The created dataset ctown.zip should be located in the datasets folder.

Next, we can train GATRes using the command:

python train.py --model gatres_small --dataset_paths "datasets/ctown.zip" --input_paths "inputs/ctown.inp" --batch_size 32

Installation

You can follow the instructions to run the data generation tool and GATRes models.

  1. Clone or download this repository.

  2. Activate venv environment with available Python (>=3.9), PyTorch (>=2.0), PyTorch Geometric (>=2.3), and/or CUDA (11.8). In case of a new environment, consider installing the required libraries via this command:

    pip install -r requirement.txt

Repo map

    |-- checkpoints                 - dir for storing model checkpoints #created in dataset generation process
    |-- configs                     - config storage for data generation tool  
    |-- experiment_logs             - dir for tracking experiment logs #created for loading model weights
    |-- generator                   - code for data generation tool
    |-- inputs                      - water distribution network topologies
    |-- utils                       - auxiliary functions for running models
    |-- ConfigModels.py             - default model configurations for running models
    |-- GraphModels.py              - model definition
    |-- evaluation.py               - code for testing model
    |-- train.py                    - code for training model

Data generation tool

This section will give you an instruction to generate an example dataset. A dataset is a set of arbitrary snapshots, an instantaneous water network state at a particular time. A state can consist of one (some) measurement, such as pressures, heads, demands, etc... To create snapshots, run the following steps:

  1. Add a preferred input file (.INP) that contains a topology into the inputs folder. Note: Skip this step if you use default public water networks in the folder (under public licenses).

  2. Add a data generation config (.INI) file. User can create a dummy one using this command:

python generator\EPYNET\ConfigCreator.py -r inputs\<your_inp_path>

You can find the created config file (.INI) in the configs folder.

  1. Run this command to create a dataset:
python generator\EPYNET\scenegenv7.py   --config <your_ini_path>
                                        --executors <num_of_workers>
                                        --batch_size <batch_size_per_worker>

There are many flags and options introduced in scenegenv7.py.

The new dataset will be a zip file (.ZIP).

Training model

User can train GATRes model using this default command:

python train.py --model gatres_small
                --epochs 500
                --batch_size 8
                --device 'cuda'
                --mask_rate 0.95
                --datasets_paths <dataset_path1> <optional_dataset_path2> ...
                --input_paths <inp_path1> <optional_inp_path2> ...
                --test_data_path <test_dataset_path>
                --test_input_path <test_dataset_path>
                --save_path <where_to_save_checkpoints>

Inference

When you run the above command, the trained model will be tested immediately after training. Otherwise, you can make a manual test using this command:

python evaluation.py    --model gatres_small
                        --model_path <where_to_load_trained_weights>
                        --batch_size 8
                        --num_test_trials 10
                        --test_type 'clean'
                        --device 'cuda'
                        --mask_rate 0.95
                        --datasets_paths <train_dataset_path> 
                        --input_paths <train_inp_path>
                        --test_data_path <test_dataset_path>
                        --test_input_path <test_dataset_path>

License

MIT license. See the LICENSE file for more details.

Citing

Please cite the following paper if you use the GATRes model or the Data Generation Tool in your research:

@misc{truong2023graph,
    title={Graph Neural Networks for Pressure Estimation in Water Distribution Systems}, 
    author={Huy Truong and Andr\'{e}s Tello and Alexander Lazovik and Victoria Degeler},
    year={2023},
    eprint={2311.10579},
    archivePrefix={arXiv},
    primaryClass={cs.LG},
    note = {submitted to Water Resources Research}
}

gnn-pressure-estimation's People

Contributors

atello avatar cuongth95 avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

haldate-yu

gnn-pressure-estimation's Issues

Getting System error: No module named 'TokenGeneratorByRange' when generating data

Hi

I am running the code on Ubuntu and after fixing some hardcoded paths, I was able to execute python generator/EPYNET/ConfigCreator.py -r inputs/CTOWN.inp

python generator/EPYNET/scenegenv7.py --config gnn-pressure-estimation/config/v7.1/CTOWN_7v1__EPYNET_config.ini

it gives ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange' error!

Here is the full stack trace:

Start simulation...
saved_path =  datasets/CTOWN
skip nodes = ['R1']
#skip_nodes = 1
#skip_links = 0
exepected #result_nodes = 395 | Note that if attribute is 'demand', #results_nodes should be #junctions
exepected #result_links = 444
  0%|                                                                                                                                              | 0/20 [00:00<?, ?it/s]/home/sug/anaconda3/envs/gnn/lib/python3.9/site-packages/wntr/epanet/io.py:2076: UserWarning: Not all curves were used in "/home/sug/gnn-pressure-estimation/inputs/CTOWN.inp"; added with type None, units conversion left to user
  warnings.warn('Not all curves were used in "{}"; added with type None, units conversion left to user'.format(self.wn.name))
/home/sug/anaconda3/envs/gnn/lib/python3.9/site-packages/epynet/baseobject.py:63: UserWarning: requesting dynamic properties from an unsolved network
  warnings.warn("requesting dynamic properties from an unsolved network")
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00,  7.02it/s]
OK
2024-03-11 20:53:38,588	INFO worker.py:1553 -- Started a local Ray instance.
  0%|                                                                                                                                              | 0/20 [00:00<?, ?it/s]WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
  5%|██████▋                                                                                                                               | 1/20 [00:00<00:06,  3.10it/s]WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341636, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f47215d2550>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
WARNING! Ray error The actor died because of an error raised in its creation task, ray::WDNRayExecutor.__init__() (pid=1341637, ip=192.168.1.82, repr=<Executorv7.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f2b6d7a55b0>)
  At least one of the input arguments for this task could not be computed:
ray.exceptions.RaySystemError: System error: No module named 'TokenGeneratorByRange'
traceback: Traceback (most recent call last):
ModuleNotFoundError: No module named 'TokenGeneratorByRange'
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:00<00:00, 60.78it/s]

Simulation time: 5.050640344619751 seconds
Process run on 20 batches, total scenes: 100
Success/Expected: 0/10 scenes

Any help on what is causing the problem?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.