Code of Paper "Joint Task Offloading and Resource Optimization in NOMA-based Vehicular Edge Computing: A Game-Theoretic DRL Approach", JSA 2022.

Home Page: https://www.sciencedirect.com/science/article/abs/pii/S138376212200265X

License: GNU General Public License v3.0

Python 100.00%

deep-reinforcement-learning noma potential-game resource-allocation task-offloading vehicular-networks

game-theoretic-deep-reinforcement-learning's Introduction

Game-Theoretic-Deep-Reinforcement-Learning

This is the code of paper, named "Joint Task Offloading and Resource Optimization in NOMA-based Vehicular Edge Computing: A Game-Theoretic DRL Approach", and the proposed solution and comparison algorithms are implemented.

Environment

The conda environment file is located in environment.yml.
It can be used to create the environment by:

conda env create -f environment.yml

File Structure

Main Function

The main() function of the repo is located in Experiment/experiment.py.

Algorithms

Multi-agent distributed distributional deep deterministic policy gradient (MAD4PG): Experiment/run_mad4pg.py
Multi-agent deep deterministic policy gradient (MADDPG): Experiment/run_maddpg.py
Distributed distributional deep deterministic policy gradient (D4PG): Experiment/run_d4pg.py
Optimal resource allocation and task local processing only (ORL): Experiment/run_optres_local.py
Optimal resource allocation and task migration only (ORM): Experiment/run_optres_edge.py

Didi Dataset

The vehicular trajectories for November 16, 2016, generated in Chengdu and extracted from the Didi GAIA Open Data Set, can be found on neardws/Vehicular-Trajectories-Processing-for-Didi-Open-Data.

Citing this paper

@article{xu2022joint,
  title={Joint task offloading and resource optimization in NOMA-based vehicular edge computing: A game-theoretic DRL approach},
  author={Xu, Xincao and Liu, Kai and Dai, Penglin and Jin, Feiyu and Ren, Hualing and Zhan, Choujun and Guo, Songtao},
  journal={Journal of Systems Architecture},
  pages={102780},
  year={2022},
  issn = {1383-7621},
  doi = {https://doi.org/10.1016/j.sysarc.2022.102780},
  url = {https://www.sciencedirect.com/science/article/pii/S138376212200265X},
  publisher={Elsevier}
}

game-theoretic-deep-reinforcement-learning's People

Contributors

Stargazers

Watchers

game-theoretic-deep-reinforcement-learning's Issues

I have faced an issue while I run the code

我在运行代码时遇到错误

亲爱的作者们，您好：

我正在读你的论文，给我留下了深刻的印象。但是当我尝试运行代码时出现此错误。
请帮助我解决这个错误
@neardws

(environment) teddy@teddy-HP-Notebook:/Desktop/Game-Theoretic-Deep-Reinforcement-Learning-main/Experiment$ python run_d4pg.py
2023-10-12 02:51:53.432688: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-10-12 02:51:53.432719: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
File "/home/teddy/Desktop/Game-Theoretic-Deep-Reinforcement-Learning-main/Experiment/run_d4pg.py", line 5, in
from Agents.D4PG_new.agent import D4PG
File "/home/teddy/Desktop/Game-Theoretic-Deep-Reinforcement-Learning-main/Agents/D4PG_new/agent.py", line 24, in
from acme import datasets
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/acme/datasets/init.py", line 18, in
from acme.datasets.reverb import make_reverb_dataset
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/acme/datasets/reverb.py", line 22, in
from acme.adders import reverb as adders
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/acme/adders/reverb/init.py", line 20, in
from acme.adders.reverb.base import DEFAULT_PRIORITY_TABLE
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/acme/adders/reverb/base.py", line 28, in
import reverb
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/reverb/init.py", line 27, in
from reverb import item_selectors as selectors
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/reverb/item_selectors.py", line 19, in
from reverb import pybind
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/reverb/pybind.py", line 6, in
_load_op_library.reraise_wrapped_error(e)
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/reverb/platform/default/load_op_library.py", line 47, in reraise_wrapped_error
raise error
File "/home/teddy/anaconda3/envs/environment/lib/python3.9/site-packages/reverb/pybind.py", line 4, in
from .libpybind import *
ImportError: libpython3.9.so.1.0: cannot open shared object file: No such file or directory
(environment) teddy@teddy-HP-Notebook:/Desktop/Game-Theoretic-Deep-Reinforcement-Learning-main/Experiment$

result files

Hello, could you provide the result files in the acme directory, such as logs.csv, because I want to compare my running results with yours. Thanks a lot.

通过yml文件创建环境

Could you please upload the dataset

Could you please upload the dataset:"DiDi Chuxing GAIA Open Dataset Initiative", (like gps_20161116.csv), which is needed in "process_trajectories.py". The website "https://outreach.didichuxing.com/research/opendata/en/" is not availabe now. Thank you.

显式解

作者您好，您论文内的Computation Resource Allocation 的kkt最优解是怎么推出来的？好像显式解推不出来吧？希望您的回复

Single GPU Environment Support

hello,I want to know if Idon`t have two GPU,can I run it?

libserialization_cc_proto.so: undefined symbol: _ZNK6google8protobuf7Message11GetTypeNameEv

您好，请问这个问题是因为啥？我查到可能是版本问题，我安装了protobuf3.17.2 ,也不可以，请问您使用的是啥版本？或许有其他解决办法吗？非常感谢🙇‍

请求仿真实验代码的一些疑惑解答

您好，非常感学您开源如此优秀的代码提供给我们学习，我是做UAV对固定物联网设备任务卸载的，想学习您的代码和**，并形成自己的仿真环境和实验，但是由于能力有限，看代码有很多的疑惑，若能解释一下不胜感激。

vehicle的trajectories_file_name，具体的数据结构是如何，是怎么封装的
在代码reaed_vehicle_trajectories中new_df = df[df['vehicle_id'] == vehicle_id]获取到对应vehicle_id的所有数据，型获取longitude和latitude列中的最大值与最小值，那么在计算max(min) distance 是否是错误的，因为获取到的(x,y)不是一个时间下的坐标。
在代码reaed_vehicle_trajectories中最大距离和最小距离计算时减去的是edge的通信范围吗，最大最小距离是到原点的距离吗？
self._vehicle_number_in_edge = int(self._vehicle_number / self._edge_number)指的是什么，vehicle与edge的比值吗，还是指的在t时刻下edge e内有x个vehicle
问题4中self._vehicle_number_in_edge与environment.py中vehicle_index_within_edges是不是同一个意思
environment.py与make_environment.py有什么区别吗，它们的作用是什么

code help

您好，我正在学习代码，想请教一个问题。就是先有了任务卸载的决策，然后进行通信资源分配，然后还有完成有线传输后，才能进行计算任务。但是假设任务k在时隙t要卸载到边缘节点e上，t时隙分配给k的通信资源为0，所以上传时延为punish_time，还要加上任务的有线传输时延，但是如果节点e又存在可用的计算资源分配，计算时延可以计算出来，但是对于边缘节点e来说，被这个车辆占用的资源持续时隙occupied_time在代码中写的是从卸载的时隙t开始到计算完成的时隙，但是不是还存在任务上传和卸载的时延吗？

Could not load dynamic library 'libcudart.so.11.0'

Windows运行环境

你好，我还想问一下这个可以在windows上面跑吗？我配置yml文件，也出现很多库下载不了的问题“Could not find a version that satisfies the requirement absl (from versions:），这个原因是因为我没在linux下面运行程序吗

Deprecation

No module named "acme.types" issue

hello,I want to konw why I have the wrong"No module named 'acme.types'",but I can't solve it,it will be wrong
"ERROR: Could not find a version that satisfies the requirement acme_types (from versions: none)
ERROR: No matching distribution found for acme_types".thank you

执行DDPG算法时报错：valueerror: typeerror: object of type 'discretevalueddistribution' has no len()

IndexError: list index out of range

你好，我在运行experiment.py时弹出这个错误，请问是为什么？
Traceback (most recent call last):
File "D:\hgxdm\dm\Game-Theoretic-Deep-Reinforcement-Learning-main\Game\Experiment\experiment.py", line 7, in
tf.config.experimental.set_virtual_device_configuration(gpus[0],
IndexError: list index out of range

关于动态库的问题

我是在执行 python expriment.py时遇到这个问题
我想请教下这个动态库未定义应该怎么办已经配置好久环境了，ku
网上查的方式都试过了都不太行

亲爱的作者您好，请问论文中边缘节点的位置是如何设置的呢？

作者您好，论文中Table 1中 𝑙𝑒 Location of edge node 𝑒的值是如何设置的呢？用的是数据集中的数据吗？谢谢！

Could not load dynamic library 'libcudart.so.11.0

博主我想问您一下您论文中的VEC架构的图是怎么画的呀，能教一下方法嘛

博主你论文里面VEC架构图是用什么画的哇能教一下方法嘛

关于acme以及sonnet等库

作者您好！最近重新拜读了您的论文和源码，由于我是一个深度学习和强化学习方面的菜鸟，所以对acme库的相关框架使用起来比较困难。
请问您有一些学习资料推荐吗？或者是否可以请教一下您的学习路线呢？

E: Unable to locate package libpython3.9

你好，请问使用sudo apt-get install libpython3.9仍旧报错是什么问题呢，谢谢谢谢

图5、6、7、8是怎么运行出来的呢

请问作者我改动了环境，是不是得通过make_environment.py文件创建新的pkl文件？但是我看文件里没有下面源码对应的trajectories_file。。

运行experiment.py时报错ModuleNotFoundError: No module named 'Experiment'

运行experiment.py时报错ModuleNotFoundError: No module named 'Experiment'
请问各位大佬 Experiment模块怎么弄哇QAQ

数据

请问原始数据集是什么呀？包括task，vehicle，edge等信息封装成的pkl文件，如果想改动代码的environment的话，请问是需要重构数据集吗？

博主能不能讲一下代码结构啊，能力不够看不太懂

请求论文和实验的一些疑惑解答

博主您好，我是一名在读研究生，十分有幸读到了您的这篇论文，想深入学习下去，在论文和实验中我遇到了一些困惑，尽力查阅了不少资料后还是不太明白，希望能够向您请教，感激不尽。
1.文中每个边缘节点的效用函数，都是所有边缘节点的服务比之和吗

2.您的势函数是如何构造出来的，不太明白势函数的第二项Ue（-Se，S-e）的意思，和怎么计算的

3.您在证明这个势函数的时候，最后一步是把Ue（-Se，S-e）和Ue（-Se‘，S-e）抵消了吗，有点不太明白怎么消去的

4.在跑您实验中的D4PG算法时，我得到的average_execution_times和average_service_times的值非常小，

使用是默认的global数据，和文中的结果不太符合，文中的D4pg，APT和AST的值看起来是7和10左右。其余的算法都符合您图中的数值，没有找到问题是出现在哪里了。
5.关于势博弈您有一些资料推荐吗，看了原文感觉读起来有些吃力。。
希望博主能在闲暇之余给予一些指导，非常感激您