Dear prosysscience:
I'm currently using this code for our work. I want to know how to generate the Gantt Chart with GIFs format like the gantt chart in your README.md.
Thank you in advance for answering the questions!

I will switch to RLLib, their implementation of PPO is way better than mine

Dear maintainers,

Thank you for this research and making the code available to the public.

I'm currently using this model for my thesis and was wondering two things;

1. How can the trained model be used later on? Are parameters saved somewhere?
2. Is it possible to train on multiple instances? Right now this is in the config: 'instance_path': 'instances/ta41', which seems to me like it only trains on that one. Is this true?

Thank you in advance for answering the questions. Have a nice day!

Can I use debug to see the action? Where should I put the break point? I put the break point in the JSS environment, but it didn't work.

For others who may encounter problems with the codecov requirement:
It is defacto removed from pypi, see https://about.codecov.io/blog/message-regarding-the-pypi-package/.
Version 2.1.13 was uploaded again, but generally the package is depricated.

PPO seems to struggle, maybe DQN will do a better jobs

Hi,
when I am running the main.py code, I encounter the following error: Exception has occurred: ModuleNotFoundError
No module named 'tree'
File "\RL-Job-Shop-Scheduling-master\JSS\CustomCallbacks.py", line 3, in
from ray.rllib.env import BaseEnv
File "\RL-Job-Shop-Scheduling-master\JSS\main.py", line 14, in
from CustomCallbacks import *
ModuleNotFoundError: No module named 'tree'.
Not all of them which import from ray.rllib, I have the same issue and it shows that module tree is not found.

Hi! I‘m new to RL and when I set up my environment and run main.py, the error is RuntimeError: Couldn't start Redis. Check log files. I wonder if it is because my redis: no config file specified, using the default config.

Automatic action when only one available cause trouble

No-Op

First observation

Having the full problem (always allow No-Op when next timestep available) makes the problem to complex

Removed

If we remove the No-Op, we can automatically perform an action when it's the only action available ====> not optimal
But it helps to remove certain features (when next need machine available), so easier representation

Outlook

Reintroduce No-Op, but only when we have one action available
Reintroduce the next-needed machine available, maybe we can mix it with the legal actions?
Keep track of him in the step, don't do like before a full compute each time!

Hi authors,

Thank you for writing such an innovative paper, I really enjoyed the detailed elaborations :)

I just wanted to know how to modify the main.py such that I'm able to output the solution schedule, and eventually render the Gantt chart based on that solution. I understand that the gym.Wrapper class, in particular, BestActionsWrapper is there in the env_wrapper.py, but how do I integrate that into main.py. Pardon my newbie question, and thank you in advance!

I am not so familiar with Ray, when I run the main.py, I got this output, is this correct? Should I wait?

Hi, sorry for bothering you again. Your JSSEnv is very helpful to my final year project, thanks!
Now I want to make comparison of FIFO with other algorithms. However, every time I run FIFO_worker or MTWR_worker, the program just ends up with only 1 episode. I'd appreciate it if you could help me figure it out. Thank you in advance. :)

Thanks a lot for providing such a nice framework to enable the job scheduling. However, when I tried to run your code with main.py and install the requirements. I found that some of the versions of the libraries didn't match. I spent a lot of time to match all the versions and now it seemed to work.

But I can't find the environment at all even if I refer to your answer here
#28. Based on your answers, I added these lines in main.py

gym.envs.register(
id='JSSEnv:jss-v1',
entry_point='JSSEnv:JSSEnv'
)

This is what I got.

/home/fan/anaconda3/bin/conda run -n JSSEnv-master --no-capture-output python /mnt/d/OneDrive - The University of Nottingham/ESR1/work/Job Shop scheduling/RL-Job-Shop-Scheduling/JSS/main.py

Starting Ray
<module 'tensorflow' from '/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/tensorflow/init.py'>
{'real_obs': array([[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.],
[1., 0., 0., 0., 0., 0., 0.]]), 'action_mask': array([ True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, False])}
wandb: Currently logged in as: univeristynottingham (use wandb login --relogin to force relogin)
wandb: wandb version 0.13.7 is available! To upgrade, please run:
wandb: $ pip install wandb --upgrade

wandb: Tracking run with wandb version 0.10.22
wandb: Syncing run legendary-lion-50
wandb: ⭐️ View project at https://wandb.ai/univeristynottingham/uncategorized
wandb: 🚀 View run at https://wandb.ai/univeristynottingham/uncategorized/runs/hyb4ba40
wandb: Run data is saved locally in /mnt/d/OneDrive - The University of Nottingham/ESR1/work/Job Shop scheduling/RL-Job-Shop-Scheduling/JSS/wandb/run-20230103_170136-hyb4ba40
wandb: Run wandb offline to turn off syncing.
2023-01-03 17:01:44,793 INFO trainer.py:2332 -- Your framework setting is 'tf', meaning you are using static-graph mode. Set framework='tf2' to enable eager execution with tf2.x. You may also then want to set eager_tracing=True in order to reach similar execution speed as with static-graph mode.
2023-01-03 17:01:44,794 WARNING deprecation.py:46 -- DeprecationWarning: simple_optimizer has been deprecated. This will raise an error in the future!
2023-01-03 17:01:44,795 WARNING deprecation.py:46 -- DeprecationWarning: metrics_smoothing_episodes has been deprecated. Use metrics_num_episodes_for_smoothing instead. This will raise an error in the future!
2023-01-03 17:01:44,795 WARNING ppo.py:386 -- train_batch_size (33000) cannot be achieved with your other settings (num_workers=16 num_envs_per_worker=4 rollout_fragment_length=704)! Auto-adjusting rollout_fragment_length to 515.
2023-01-03 17:01:44,795 INFO ppo.py:414 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.
2023-01-03 17:01:44,795 INFO trainer.py:903 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
(pid=5105)
(pid=5101)
(pid=5107)
(pid=5103)
(pid=5100)
(pid=5109)
(pid=5111)
(pid=5099)
(pid=5112)
(pid=5098)
(pid=5108)
(pid=5102)
(pid=5110)
(pid=5579)
(pid=5582)
(pid=5580)
(pid=5104)
(pid=5106)
(pid=5574)
(pid=5578)
(pid=5097)
(pid=5577)
(pid=5576)
(pid=5581)
(pid=5584)
(pid=5585)
(pid=5583)
(pid=5587)
(pid=5575)
(pid=5589)
(pid=5588)
(pid=5586)
(RolloutWorker pid=5576) 2023-01-03 17:01:53,202 ERROR worker.py:451 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=5576, ip=172.30.112.110, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f9485a6ce80>)
(RolloutWorker pid=5576) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 676, in make
(RolloutWorker pid=5576) return registry.make(id, **kwargs)
(RolloutWorker pid=5576) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 490, in make
(RolloutWorker pid=5576) versions = self.env_specs.versions(namespace, name)
(RolloutWorker pid=5576) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 220, in versions
(RolloutWorker pid=5576) self._assert_name_exists(namespace, name)
(RolloutWorker pid=5576) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 297, in _assert_name_exists
(RolloutWorker pid=5576) raise error.NameNotFound(message)
(RolloutWorker pid=5576) gym.error.NameNotFound: Environment jss doesn't exist.
(RolloutWorker pid=5576)
(RolloutWorker pid=5576) During handling of the above exception, another exception occurred:
(RolloutWorker pid=5576)
(RolloutWorker pid=5576) ray::RolloutWorker.init() (pid=5576, ip=172.30.112.110, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f9485a6ce80>)
(RolloutWorker pid=5576) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 506, in init
(RolloutWorker pid=5576) self.env = env_creator(copy.deepcopy(self.env_context))
(RolloutWorker pid=5576) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/ray/rllib/env/utils.py", line 56, in gym_env_creator
(RolloutWorker pid=5576) raise EnvError(ERR_MSG_INVALID_ENV_DESCRIPTOR.format(env_descriptor))
(RolloutWorker pid=5576) ray.rllib.utils.error.EnvError: The env string you provided ('jss-v1') is:
(RolloutWorker pid=5576) a) Not a supported/installed environment.
(RolloutWorker pid=5576) b) Not a tune-registered environment creator.
(RolloutWorker pid=5576) c) Not a valid env class string.
(RolloutWorker pid=5576)
(RolloutWorker pid=5576) Try one of the following:
(RolloutWorker pid=5576) a) For Atari support: pip install gym[atari] autorom[accept-rom-license].
(RolloutWorker pid=5576) For VizDoom support: Install VizDoom
(RolloutWorker pid=5576) (https://github.com/mwydmuch/ViZDoom/blob/master/doc/Building.md) and
(RolloutWorker pid=5576) pip install vizdoomgym.
(RolloutWorker pid=5576) For PyBullet support: pip install pybullet.
(RolloutWorker pid=5576) b) To register your custom env, do from ray import tune; (RolloutWorker pid=5576) tune.register('[name]', lambda cfg: [return env obj from here using cfg]).
(RolloutWorker pid=5576) Then in your config, do config['env'] = [name].
(RolloutWorker pid=5576) c) Make sure you provide a fully qualified classpath, e.g.:
(RolloutWorker pid=5576) ray.rllib.examples.env.repeat_after_me_env.RepeatAfterMeEnv
(RolloutWorker pid=5584) 2023-01-03 17:01:53,224 ERROR worker.py:451 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=5584, ip=172.30.112.110, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f26b16a8e80>)
(RolloutWorker pid=5584) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 676, in make
(RolloutWorker pid=5584) return registry.make(id, **kwargs)
(RolloutWorker pid=5584) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 490, in make
(RolloutWorker pid=5584) versions = self.env_specs.versions(namespace, name)
(RolloutWorker pid=5584) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 220, in versions
(RolloutWorker pid=5584) self._assert_name_exists(namespace, name)
(RolloutWorker pid=5584) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 297, in _assert_name_exists
(RolloutWorker pid=5584) raise error.NameNotFound(message)
(RolloutWorker pid=5584) gym.error.NameNotFound: Environment jss doesn't exist.
(RolloutWorker pid=5584)
(RolloutWorker pid=5584) During handling of the above exception, another exception occurred:
(RolloutWorker pid=5584)
(RolloutWorker pid=5584) ray::RolloutWorker.init() (pid=5584, ip=172.30.112.110, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f26b16a8e80>)
(RolloutWorker pid=5584) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 506, in init
(RolloutWorker pid=5584) self.env = env_creator(copy.deepcopy(self.env_context))
(RolloutWorker pid=5584) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/ray/rllib/env/utils.py", line 56, in gym_env_creator
(RolloutWorker pid=5584) raise EnvError(ERR_MSG_INVALID_ENV_DESCRIPTOR.format(env_descriptor))
(RolloutWorker pid=5584) ray.rllib.utils.error.EnvError: The env string you provided ('jss-v1') is:
(RolloutWorker pid=5584) a) Not a supported/installed environment.
(RolloutWorker pid=5584) b) Not a tune-registered environment creator.
(RolloutWorker pid=5584) c) Not a valid env class string.
(RolloutWorker pid=5584)
(RolloutWorker pid=5584) Try one of the following:
(RolloutWorker pid=5584) a) For Atari support: pip install gym[atari] autorom[accept-rom-license].
(RolloutWorker pid=5584) For VizDoom support: Install VizDoom
(RolloutWorker pid=5584) (https://github.com/mwydmuch/ViZDoom/blob/master/doc/Building.md) and
(RolloutWorker pid=5584) pip install vizdoomgym.
(RolloutWorker pid=5584) For PyBullet support: pip install pybullet.
(RolloutWorker pid=5584) b) To register your custom env, do from ray import tune; (RolloutWorker pid=5584) tune.register('[name]', lambda cfg: [return env obj from here using cfg]).
(RolloutWorker pid=5584) Then in your config, do config['env'] = [name].
(RolloutWorker pid=5584) c) Make sure you provide a fully qualified classpath, e.g.:
(RolloutWorker pid=5584) ray.rllib.examples.env.repeat_after_me_env.RepeatAfterMeEnv
(RolloutWorker pid=5579) 2023-01-03 17:01:53,340 ERROR worker.py:451 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=5579, ip=172.30.112.110, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f858809ce80>)
(RolloutWorker pid=5579) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 676, in make
(RolloutWorker pid=5579) return registry.make(id, **kwargs)
(RolloutWorker pid=5579) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 490, in make
(RolloutWorker pid=5579) versions = self.env_specs.versions(namespace, name)
(RolloutWorker pid=5579) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 220, in versions
(RolloutWorker pid=5579) self._assert_name_exists(namespace, name)
(RolloutWorker pid=5579) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 297, in _assert_name_exists
(RolloutWorker pid=5579) raise error.NameNotFound(message)
(RolloutWorker pid=5579) gym.error.NameNotFound: Environment jss doesn't exist.
(RolloutWorker pid=5579)
(RolloutWorker pid=5579) During handling of the above exception, another exception occurred:
(RolloutWorker pid=5579)
(RolloutWorker pid=5579) ray::RolloutWorker.init() (pid=5579, ip=172.30.112.110, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f858809ce80>)
(RolloutWorker pid=5579) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 506, in init
(RolloutWorker pid=5579) self.env = env_creator(copy.deepcopy(self.env_context))
(RolloutWorker pid=5579) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/ray/rllib/env/utils.py", line 56, in gym_env_creator
(RolloutWorker pid=5579) raise EnvError(ERR_MSG_INVALID_ENV_DESCRIPTOR.format(env_descriptor))
(RolloutWorker pid=5579) ray.rllib.utils.error.EnvError: The env string you provided ('jss-v1') is:
(RolloutWorker pid=5579) a) Not a supported/installed environment.
(RolloutWorker pid=5579) b) Not a tune-registered environment creator.
(RolloutWorker pid=5579) c) Not a valid env class string.
(RolloutWorker pid=5579)
(RolloutWorker pid=5579) Try one of the following:
(RolloutWorker pid=5579) a) For Atari support: pip install gym[atari] autorom[accept-rom-license].
(RolloutWorker pid=5579) For VizDoom support: Install VizDoom
(RolloutWorker pid=5579) (https://github.com/mwydmuch/ViZDoom/blob/master/doc/Building.md) and
(RolloutWorker pid=5579) pip install vizdoomgym.
(RolloutWorker pid=5579) For PyBullet support: pip install pybullet.
(RolloutWorker pid=5579) b) To register your custom env, do from ray import tune; (RolloutWorker pid=5579) tune.register('[name]', lambda cfg: [return env obj from here using cfg]).
(RolloutWorker pid=5579) Then in your config, do config['env'] = [name].
(RolloutWorker pid=5579) c) Make sure you provide a fully qualified classpath, e.g.:
(RolloutWorker pid=5579) ray.rllib.examples.env.repeat_after_me_env.RepeatAfterMeEnv
(RolloutWorker pid=5582) 2023-01-03 17:01:53,322 ERROR worker.py:451 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=5582, ip=172.30.112.110, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7f36879d4e50>)
(RolloutWorker pid=5582) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 676, in make
(RolloutWorker pid=5582) return registry.make(id, **kwargs)
(RolloutWorker pid=5582) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 490, in make
(RolloutWorker pid=5582) versions = self.env_specs.versions(namespace, name)
(RolloutWorker pid=5582) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 220, in versions
(RolloutWorker pid=5582) self._assert_name_exists(namespace, name)
(RolloutWorker pid=5582) File "/home/fan/anaconda3/envs/JSSEnv-master/lib/python3.10/site-packages/gym/envs/registration.py", line 297, in _assert_name_exists
(RolloutWorker pid=5582) raise error.NameNotFound(message)
(RolloutWorker pid=5582) gym.error.NameNotFound: Environment jss doesn't exist.
(RolloutWorker pid=5582)
(RolloutWorker pid=5582) During handling of the above exception, another exception occurred:
(RolloutWorker pid=5582)
(RolloutWorker pid=5582) ray::RolloutWorker.init() (pid=5582, ip=172.30.112.110, repr=

Process finished with exit code 1

First of all I would like to thank you for doing the implementation of the idea you proposed in your paper and also for such a real nice paper. Learnt a lot from it!
I am doing research in similar area and your paper has been a great starting point.

I am using Ubuntu 21.10 and Python version 3.9
While running the code I found lot of errors while installing the packages listed in requirements.txt because of the dependencies and deprecated versions so
and that makes sense because now lot of newer version of these packages are there and the older ones do no support so I tried installing the packages listed in requirements.txt by removing all the versions in there and it did install successfully.

But then somehow the JSSEnv is not available although I did install it.

Here is the error that I am getting and somehow I am stuck here, Any help will be appreciated alot.

wandb: Currently logged in as: smita09 (usewandb login --reloginto force relogin) wandb: Tracking run with wandb version 0.12.11 wandb: Run data is saved locally in /home/smita/RL-Job-Shop-Scheduling/JSS/wandb/run-20220304_054910-393en4ar wandb: Runwandb offlineto turn off syncing. wandb: Syncing run worldly-flower-21 wandb: ⭐️ View project at https://wandb.ai/smita09/RL-Job-Shop-Scheduling-JSS wandb: 🚀 View run at https://wandb.ai/smita09/RL-Job-Shop-Scheduling-JSS/runs/393en4ar E0304 05:49:15.560678575 29964 fork_posix.cc:70] Fork support is only compatible with the epoll1 and poll polling strategies E0304 05:49:15.955048134 29964 fork_posix.cc:70] Fork support is only compatible with the epoll1 and poll polling strategies 2022-03-04 05:49:17,754 INFO services.py:1374 -- View the Ray dashboard at http://127.0.0.1:8265 E0304 05:49:17.774863530 29964 fork_posix.cc:70] Fork support is only compatible with the epoll1 and poll polling strategies E0304 05:49:17.810299565 29964 fork_posix.cc:70] Fork support is only compatible with the epoll1 and poll polling strategies 2022-03-04 05:49:21,270 INFO trainer.py:2054 -- Your framework setting is 'tf', meaning you are using static-graph mode. Set framework='tf2' to enable eager execution with tf2.x. You may also then want to set eager_tracing=True in order to reach similar execution speed as with static-graph mode. 2022-03-04 05:49:21,281 WARNING deprecation.py:45 -- DeprecationWarning:simple_optimizerhas been deprecated. This will raise an error in the future! 2022-03-04 05:49:21,282 WARNING deprecation.py:45 -- DeprecationWarning:metrics_smoothing_episodeshas been deprecated. Usemetrics_num_episodes_for_smoothinginstead. This will raise an error in the future! 2022-03-04 05:49:21,282 WARNING deprecation.py:45 -- DeprecationWarning:metrics_smoothing_episodeshas been deprecated. Usemetrics_num_episodes_for_smoothinginstead. This will raise an error in the future! 2022-03-04 05:49:21,282 WARNING ppo.py:223 --train_batch_size(33000) cannot be achieved with your other settings (num_workers=2 num_envs_per_worker=4 rollout_fragment_length=704)! Auto-adjustingrollout_fragment_length` to 4125.
2022-03-04 05:49:21,282 INFO ppo.py:249 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.
2022-03-04 05:49:21,282 INFO trainer.py:790 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
(pid=30192)
(pid=30190)
Traceback (most recent call last):
File "/home/smita/second_env/lib/python3.9/site-packages/ray/rllib/agents/trainer.py", line 811, in setup
self._init(self.config, self.env_creator)
File "/home/smita/second_env/lib/python3.9/site-packages/ray/rllib/agents/trainer.py", line 923, in _init
raise NotImplementedError
NotImplementedError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/smita/RL-Job-Shop-Scheduling/JSS/main.py", line 161, in
train_func()
File "/home/smita/RL-Job-Shop-Scheduling/JSS/main.py", line 148, in train_func
trainer = PPOTrainer(config=config)
File "/home/smita/second_env/lib/python3.9/site-packages/ray/rllib/agents/trainer.py", line 728, in init
super().init(config, logger_creator, remote_checkpoint_dir,
File "/home/smita/second_env/lib/python3.9/site-packages/ray/tune/trainable.py", line 122, in init
self.setup(copy.deepcopy(self.config))
File "/home/smita/second_env/lib/python3.9/site-packages/ray/rllib/agents/trainer.py", line 826, in setup
self.workers = self._make_workers(
File "/home/smita/second_env/lib/python3.9/site-packages/ray/rllib/agents/trainer.py", line 1925, in _make_workers
return WorkerSet(
File "/home/smita/second_env/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 100, in init
remote_spaces = ray.get(self.remote_workers(
File "/home/smita/second_env/lib/python3.9/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/home/smita/second_env/lib/python3.9/site-packages/ray/worker.py", line 1735, in get
raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=30190, ip=192.168.140.128)
File "/home/smita/second_env/lib/python3.9/site-packages/gym/envs/registration.py", line 676, in make
return registry.make(id, **kwargs)
File "/home/smita/second_env/lib/python3.9/site-packages/gym/envs/registration.py", line 490, in make
versions = self.env_specs.versions(namespace, name)
File "/home/smita/second_env/lib/python3.9/site-packages/gym/envs/registration.py", line 220, in versions
self._assert_name_exists(namespace, name)
File "/home/smita/second_env/lib/python3.9/site-packages/gym/envs/registration.py", line 297, in _assert_name_exists
raise error.NameNotFound(message)
gym.error.NameNotFound: Environment JSSEnv:jss doesn't exist.

During handling of the above exception, another exception occurred:

ray::RolloutWorker.init() (pid=30190, ip=192.168.140.128)
File "/home/smita/second_env/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 460, in init
self.env = env_creator(copy.deepcopy

wandb: Waiting for W&B process to finish... (failed 1). Press Control-C to abort syncing.
wandb:
wandb: Synced worldly-flower-21: https://wandb.ai/smita09/RL-Job-Shop-Scheduling-JSS/runs/393en4ar
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20220304_054910-393en4ar/logs`

Thanks in Advance!
Have a great day! :)

Need to create a random agent in order to compare the performance with reinforcement learning

I am new to this area, and I just get the make span, but could not find the solution sequence, I need help.

Hi,
Thanks for your Awesome work. Very interesting
Can you please explain in short how do you run this entire program? I am quite new to Reinforcement learning. I can see there are 2 set of files: One Specifically for the environment and the other for the RL algorithm. I use Ubuntu 18.04 so I feel there will not be any additional requirements to be installed.
Waiting to hear from you.

I need to replace my current old Venv by the one from stable baseline (even try RLLIB ?)

Sorry for bothering. I got the "NotADirectoryError" below when running "main.py" program. However, the directory "'C:\Users\pmb433/ray_results" is well written. I wonder if you have any idea why this happened. Thanks!

2021-03-29 14:56:21,906 INFO services.py:1173 -- View the Ray dashboard at �[1m�[32mhttp://127.0.0.1:8265�[39m�[22m
Traceback (most recent call last):
File "main.py", line 161, in
train_func()
File "main.py", line 148, in train_func
trainer = PPOTrainer(config=config)
File "C:\Users\pmb433\AppData\Roaming\Python\Python37\site-packages\ray\rllib\agents\trainer_template.py", line 106, in init
Trainer.init(self, config, env, logger_creator)
File "C:\Users\pmb433\AppData\Roaming\Python\Python37\site-packages\ray\rllib\agents\trainer.py", line 465, in init
super().init(config, logger_creator)
File "C:\Users\pmb433\AppData\Roaming\Python\Python37\site-packages\ray\tune\trainable.py", line 75, in init
self._create_logger(self.config, logger_creator)
File "C:\Users\pmb433\AppData\Roaming\Python\Python37\site-packages\ray\tune\trainable.py", line 446, in _create_logger
self._result_logger = logger_creator(config)
File "C:\Users\pmb433\AppData\Roaming\Python\Python37\site-packages\ray\rllib\agents\trainer.py", line 460, in default_logger_creator
prefix=logdir_prefix, dir=DEFAULT_RESULTS_DIR)
File "C:\ProgramData\Anaconda3\lib\tempfile.py", line 366, in mkdtemp
_os.mkdir(file, 0o700)
NotADirectoryError: [WinError 267] The directory name is invalid: `'C:\Users\pmb433/ray_results\PPO_JSSEnv:jss-v1_2021-03-29_14-56-32sko0l688'

Hello,

thank you first of all for the paper and the code. I do however have a small question about the specification of the solution.
When I run main.py from the save_soltion-branch, I get a 2D-array called best_solution (main.py line 167 prints this array). But I don't really understand what numbers in this array mean.
When I run main.py with an instance, that has 2 Jobs with 15 machines, the solution looks like this:
[[ 0 94 160 193 246 272 287 352 434 444 471 564 656 752 822]
[ 0 74 105 193 244 301 434 442 449 540 619 637 688 706 805]]

At first, I thought it follows the specification of the Solution start file, but as far I understand that would mean, that both jobs start machine 1 at time 0. And also, both jobs would start at machine 4 at time 193. But a machine can only work at one job at any given time.

So I wanted to ask, how exactly should we interpret the solution?
Since I might be missing something really obvious here, I want to thank you again for your time and for the effort you put in to make your research accessible.

Kind regards,
D-Doge

Hi Pierre Tassel,

Thank you very much for sharing your implementation of RL based solution for Job Shop Scheduling Problem. This really helps students like me who are learning RL implementations that address real world problems.

While I was trying to get the code working on my pc (Windows 10 laptop), I encountered the below issue:

Exception has occurred: NotADirectoryError
[WinError 267] The directory name is invalid: 'C:\Users\pg/ray_results\PPO_JSSEnv:jss-v1_2023-01-15_08-54-17l2jg9jkq'
File "C:\Users\pg\RL-Job-Shop-Scheduling-master\JSS\main.py", line 152, in train_func
trainer = PPOTrainer(config=config)
File "C:\Users\pg\RL-Job-Shop-Scheduling-master\JSS\main.py", line 165, in
train_func()

I'm suspecting this is due to windows o/s doesnt support creating folder with a colon (:) character in the folder name, as the PPOTrainer( ) tries to create a folder with the environment name (JSSEnc:jss-v1) as a part of it.

Can you please let me know how I can address this issue? I tried without the JSSEnv: prefix, but then it is unable to recognize the environment name. Some solutions on the web talk about implementing a custom logger which doesnt seem to be straight forward. Is there a simpler workaround for this issue?

Regards,
Prasanth

Maybe the Boltzmann approach is more adapted to our problem
Because:

• our problem is deterministic
• the impact of one action can be huge
• our action space is big, even with a very big epsilon, we will still not be able to visit everything
• we work under time constraint so and we want to found goods solution during training too

Hi authors,

Thanks for the great work of the paper and code.
I have a question about the format of input instances. As you mentioned in README.md, the instances must follow Taillard's specification at http://jobshop.jjvh.nl/explanation.php#taillard_def.
But when I checked out given instances at JSS/instances/* , they are following standard specification instead.
Did the website changed the specification format or am I not understanding the instructions.

Minor but still nice enhancement for the sweep

Hi,

I am working on a method to forecast the best order for incoming jobs in an industrial factory for a university project. In our case, there are a bunch of machines where different products need to run. Your code implementation already covers a big part of my problem, but I also need to regard several set-up times of the machines. The set-up time depends on which product was produced before. With the method we are searching for the shortest duration time under regard of the set-up times and orders.
Could you please help me, how to implement this issue?

Kind regards.

I'm using mac and what I run this command, there is an ERROR "Could not install packages due to an OSError: [Errno 2] No such file or directory: '/System/Volumes/Data/home/conda/feedstock_root/build_artifacts/adal_1591523462562/work'". I wonder how to solve this problem.... Thanks in advance :)

Needs to have some unit test for the environment

First of all, I want to compliment you on this project and the paper! Thanks a lot for making it public!
However, I could not find the Python version you used for the project. I would like to know it for setting the project up as smoothly as possible.

Small bug on the presentation, fixed on the DQN branch not on the PPO branch

When I running the main.py , I get an error like this:

WARNING:tensorflow:From C:\Users\admin\anaconda3\envs\JSS0\lib\site-packages\tensorflow\python\compat\v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Traceback (most recent call last):
File "G:/文献阅读/2021-5/JSS-master/JSS/main.py", line 161, in
train_func()
File "G:/文献阅读/2021-5/JSS-master/JSS/main.py", line 104, in train_func
wandb.init(config=default_config)
File "C:\Users\admin\anaconda3\envs\JSS0\lib\site-packages\wandb\sdk\wandb_init.py", line 740, in init
wi.setup(kwargs)
File "C:\Users\admin\anaconda3\envs\JSS0\lib\site-packages\wandb\sdk\wandb_init.py", line 153, in setup
wandb_login._login(anonymous=anonymous, force=force, _disable_warning=True)
File "C:\Users\admin\anaconda3\envs\JSS0\lib\site-packages\wandb\sdk\wandb_login.py", line 210, in _login
wlogin.prompt_api_key()
File "C:\Users\admin\anaconda3\envs\JSS0\lib\site-packages\wandb\sdk\wandb_login.py", line 147, in prompt_api_key
raise UsageError("api_key not configured (no-tty). Run wandb login")
wandb.errors.error.UsageError: api_key not configured (no-tty). Run wandb login

Process finished with exit code 1

Do you know how to fix it?
Thanks.

add FIFO for comparison

Some unit test were more than needed !

Needs to clean a bit the master branch

Thank you for the great work and effort with details! I could get it working. I would like to play around changing
the number of machines and jobs and the processing time. Few clarifications I thought I ask here.

1. I realize the instance file is what is used to provide the inputs
2. Maybe the first line is number of machines and jobs
3. The other rows and columns how to interpret these?
   Thank you very much for the time and effort!

Try to have a reset strategy to counter the impact of first actions

when I run the code in master branch, I get the follow result:

how to get the result of paper: 2208

The title says it all!