cics-nd / ar-pde-cnn Goto Github PK

Physics-constrained auto-regressive convolutional neural networks for dynamical PDEs

Home Page: https://doi.org/10.1016/j.jcp.2019.109056

License: MIT License

Python 98.66% MATLAB 1.34%

physics-constrained deep-learning convolutional-neural-networks partial-differential-equations physics-informed bayesian-surrogate

ar-pde-cnn's Introduction

Physics-Constrained Auto-Regressive Convolutional Neural Networks

Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks [JCP][ArXiv]

Nicholas Geneva, Nicholas Zabaras

Highlights	Bayesian 2D Coupled Burgers' Prediction
Zero Training Data Surrogate Modeling Deep Convolutional Networks Non-linear Dynamics Chaotic Systems Bayesian Framework 500x Faster than FEM

In this work, we propose a novel auto-regressive dense encoder-decoder convolutional neural network to solve and model transient systems with non-linear dynamics at a computational cost that is potentially magnitudes lower than standard numerical solvers. This model includes a Bayesian framework that allows for uncertainty quantification of the predicted quantities of interest at each time-step. We rigorously test this model on several non-linear transient partial differential equation systems including the turbulence of the Kuramoto-Sivashinsky equation, multi-shock formation and interaction with 1D Burgers’ equation and 2D wave dynamics with coupled Burgers’ equations.

Index

Each of the PDE systems used in the paper is designated its own folder where more information can be found regarding model training, testing and figure replication.

1D-KS-SWAG: The 1D Kuramoto-Sivashinsky system.
1D-Burger-SWAG: The 1D viscous Burgers' system.
2D-Burgers-SWAG: The 2D coupled Burgers' system.

Deep Turbulence Generation

Core Dependencies

Python 3.6.5
PyTorch 1.1.0
Fenics 2019.1.0
Matplotlib 3.1.1

Plus additional standard packages such as Numpy and Scipy

Note: PyTorch <= 1.2.0 must be used due to a change padding_mode='circular' behavior, for higher PyTorch versions reduce padding to 1 (currently 2 due to a bug in PyTorch versions 1.1.0 and 1.2.0).

Citation

Find this useful or like this work? Cite us with:

@article{geneva2020modeling,
  title = {Modeling the dynamics of {PDE} systems with physics-constrained deep auto-regressive networks},
  journal = {Journal of Computational Physics},
  volume = {403},
  pages = {109056},
  year = {2020},
  issn = {0021-9991},
  doi = {10.1016/j.jcp.2019.109056},
  url = {http://www.sciencedirect.com/science/article/pii/S0021999119307612},
  author = {Nicholas Geneva and Nicholas Zabaras}
}

ar-pde-cnn's People

Contributors

Stargazers

Watchers

ar-pde-cnn's Issues

a request

Hello! In your paper, formula (19) is used to design the loss function of the model, and the corresponding code is in bayesNN.py.
But I can't understand the implementation principle in the calc_neg_log_joint function.
Could you explain the principle of formula (19) implemented in the program further?
Thank you very much!

Error from 2D Case When Running from Scratch

Hello,

I'm really interesting in your work, so I tried to run it. However, when I test the code for the 2D Burgers case, I got following errors. I guess there should be some errors for the DenseNet, particularly the size for kernel, stride and padding (especially the 'circular padding' you used). By the way, I used the dataset you provided to train it from scratch.

Thanks,

Training network, lets rock
Traceback (most recent call last):
File "main.py", line 267, in
tsteps, tback, tstart, dt)
File "main.py", line 61, in train
uPred = model(input[:,-2*args.nic:,:])
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "=/Desktop/ar-pde-cnn-master/2D-Burgers-SWAG/nn/swag.py", line 67, in forward
return self.base(*args, **kwargs)
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/Desktop/ar-pde-cnn-master/2D-Burgers-SWAG/nn/bayesNN.py", line 75, in forward
output = self.model(input)
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/Desktop/ar-pde-cnn-master/2D-Burgers-SWAG/nn/denseEDcirc2d.py", line 314, in forward
return self.features(x)
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/container.py", line 119, in forward
input = module(input)
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/container.py", line 119, in forward
input = module(input)
File "/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/Desktop/ar-pde-cnn-master/2D-Burgers-SWAG/nn/denseEDcirc2d.py", line 68, in forward
return torch.cat([x, y], 1)
RuntimeError: Sizes of tensors must match except in dimension 1. Got 35 and 37 in dimension 2 (The offending index is 1)

Error growth fast in the extrapolation region of t

Hi,

I am very interested in your work and thank you very much make the code public.

From the Figure 11 and 20 in the paper, for deterministic model, the error growth much faster in extrapolation region than in interpolation region of t and I have two questions on it:

I learnt that the model is auto-regressive, which means it does not have a sense of t, so does that mean that the model, if well trained on all the inputs, should have steadily growing error over t (in both extrapolation and interpolation region)?
I saw the error growth much faster once get into extrapolation region, could you explain why?

Thanks.

a little puzzle

I am very interested in your work. When I run this program (https://github.com/cics-nd/ar-pde-cnn/blob/master/1D-KS-SWAG/nn/denseEDcirc.py), the program output is as follows:

DenseED(
(features): Sequential(
(In_conv): Conv1d(1, 48, kernel_size=(7,), stride=(2,), padding=(6,), bias=False)
(EncBlock1): _DenseBlock(
(denselayer1): _DenseLayer(
(norm1): BatchNorm1d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(48, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(denselayer2): _DenseLayer(
(norm1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(64, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(denselayer3): _DenseLayer(
(norm1): BatchNorm1d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(80, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
)
(TransDown1): _Transition(
(norm1): BatchNorm1d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(96, 48, kernel_size=(1,), stride=(1,), bias=False)
(norm2): BatchNorm1d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace)
(conv2): Conv1d(48, 48, kernel_size=(4,), stride=(2,), padding=(2,), bias=False)
)
(DecBlock1): _DenseBlock(
(denselayer1): _DenseLayer(
(norm1): BatchNorm1d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(48, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(denselayer2): _DenseLayer(
(norm1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(64, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(denselayer3): _DenseLayer(
(norm1): BatchNorm1d(80, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(80, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(denselayer4): _DenseLayer(
(norm1): BatchNorm1d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(96, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
)
(TransUp1): _Transition(
(norm1): BatchNorm1d(112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(112, 56, kernel_size=(1,), stride=(1,), bias=False)
(norm2): BatchNorm1d(56, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace)
(upsample): UpsamplingNearest1d()
(conv2): Conv1d(56, 56, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(DecBlock2): _DenseBlock(
(denselayer1): _DenseLayer(
(norm1): BatchNorm1d(56, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(56, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(denselayer2): _DenseLayer(
(norm1): BatchNorm1d(72, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(72, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
(denselayer3): _DenseLayer(
(norm1): BatchNorm1d(88, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(88, 16, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
)
)
(LastTransUp): Sequential(
(norm1): BatchNorm1d(104, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace)
(conv1): Conv1d(104, 52, kernel_size=(1,), stride=(1,), bias=False)
(norm2): BatchNorm1d(52, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace)
(upsample): UpsamplingNearest1d()
(conv2): Conv1d(52, 26, kernel_size=(3,), stride=(1,), padding=(2,), bias=False)
(norm3): BatchNorm1d(26, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu3): ReLU(inplace)
(conv3): Conv1d(26, 1, kernel_size=(5,), stride=(1,), padding=(4,), bias=False)
)
(Tanh): Tanh()
)
)
input: torch.Size([16, 1, 512])
In_conv: torch.Size([16, 48, 256])
EncBlock1: torch.Size([16, 96, 256])
TransDown1: torch.Size([16, 48, 128])
DecBlock1: torch.Size([16, 112, 128])
TransUp1: torch.Size([16, 56, 256])
DecBlock2: torch.Size([16, 104, 256])
LastTransUp: torch.Size([16, 1, 512])
Tanh: torch.Size([16, 1, 512])
(75222, 18)

Here I have a small confusion. After the layer of In_conv, the data dimension is torch.Size([16, 48, 256]), but after the EncBlock1 layer, the dimension of the data will be torch.Size([ 16, 96, 256])?
In other words, why does the number of channels of data change from 48 to 96?
According to the meaning of the convolutional layer, I am a bit incomprehensible.
Thank you very much for answering my questions during your busy schedule.

a A small request

What principles do tsteps and tback implement in the program?
Could you give me some advice?
Thank you very much!

cics-nd / ar-pde-cnn Goto Github PK

ar-pde-cnn's Introduction

Physics-Constrained Auto-Regressive Convolutional Neural Networks

Index

Core Dependencies

Citation

ar-pde-cnn's People

Contributors

Stargazers

Watchers

Forkers

ar-pde-cnn's Issues

a request

Error from 2D Case When Running from Scratch

Error growth fast in the extrapolation region of t

a little puzzle

a A small request

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent