unity-technologies / obstacle-tower-source Goto Github PK

Obstacle Tower Source Code

License: Apache License 2.0

C# 73.91% HLSL 3.30% ShaderLab 22.79%

obstacle-tower-source's Introduction

Obstacle Tower (Source code)

The Obstacle Tower is a procedurally generated environment consisting of an endless number of floors to be solved by a learning agent. It is designed to examine how machines operate in a variety of areas, including computer vision, locomotion skills, and high-level planning. It combines platforming-style gameplay with puzzles and planning problems, all in a tower with an endless number of floors for agents to learn to solve. Critically, the floors become progressively more difficult as the agent progresses.

Within each floor, the goal of the agent is to arrive at the set of stairs leading to the next level of the tower. These floors are composed of multiple rooms, each which can contain their own unique challenges. Furthermore, each floor contains a number of procedurally generated elements, such as visual appearance, puzzle configuration, and floor layout. This ensures that in order for an agent to be successful at the Obstacle Tower task, they must be able to generalize to new and unseen combinations of conditions.

Reference paper

To learn more, please read our IJCAI 2019 paper:

Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning.

If you use Obstacle Tower in your research, we ask that you cite the paper above.

Training agents with a pre-built environment

If you are interested primarily in AI research using Obstacle Tower without the need to modify the environment itself, we provide pre-built binaries and a gym wrapper to interact with the environment. It is available here.

Requirements

Opening the project

Open Unity Editor and load project from the root of this directory.
From editor, load Procedural scene, located in Assets/ObstacleTower/Scenes.
Click on Play button in Editor to run environment with player controls.

Understanding and extending the project

To learn more about the project and how to extend Obstacle Tower for your own custom research, see here.

Obstacle Tower Challenge

On February 11, 2019, Unity Technologies launched a challenge using the Obstacle Tower. The challenge ended on July 15, 2019. See below for more information on the Obstacle Tower Challenge.

obstacle-tower-source's People

Contributors

Stargazers

Watchers

obstacle-tower-source's Issues

Shaders error in TV_Pattern.shadergrapg (TV_Pattern)

Hi @awjuliani !

The file \obstacle-tower-source\Assets\WorldBuilding\Materials\Future\TV_Pattern.shadergraph raises two Shader errors.
Due to this, the TVs' materials are rendered purple.

This issue is also observed on earlier commits.

System:
Unity 2019.2.19f
Windows 10

Shader error in 'Shader Graphs/TV_Pattern': incorrect number of arguments to numeric-type constructor at line 285 (on d3d11)

Compiling Vertex program with _MAIN_LIGHT_SHADOWS _SHADOWS_SOFT
Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_RGBM_ENCODING

Shader error in 'Shader Graphs/TV_Pattern': incorrect number of arguments to numeric-type constructor at line 808 (on d3d11)

Compiling Fragment program with UNITY_PASS_SHADOWCASTER
Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_RGBM_ENCODING

Missing sokoban platform

Hi,
I encountered a bug while running recording a demonstration. These are the initialization parameters:

name='demo1'
[reset_parameters]
tower-seed=100
starting-floor=0
dense-reward=0
lighting-type=1
visual-theme=0
agent-perspective=1
allowed-rooms=2
allowed-modules=2
allowed-floors=2
total-floors=100
default-theme=0

In floor 21, there is a bug and the sokoban platform isn't showing.

If I start over from the same floor, the platform is there.

The bug seems to depend on the previous floor because I began from 20 this time and went to 21 and it didn't show up.

Replaying a demonstration from python fails to take the same path as the demo

I have recorded some demonstrations through the unity editor and I have tried to retrace the path, i.e. take the exact same actions as those in the demo but the agent always ends up failing to reach the same point. I have initialized the environment with the same seed as that from the demo.

def replay(path, **other_params):
    brain_params, brain_infos, _ = demo_loader.load_demonstration(str(path))
    #some initialization and setting up
    # ....
    # ....
    env.reset()
    #starting from 1 because the info contains the previous action
    for binfo in brain_infos[1:]:
        #process_info extracts the vector of the previous_action
        _,_,_, info = process_info(binfo)
        _,_,_, newinfo = env.step(info.previous_action)

My goal is to be able to take in a demonstration and take the same steps as those in the demo in order to return to the same location as the final step (of the demo).

[issue] No protocol specified

Hi @awjuliani ,
I am trying to run the obstacle tower on a remote machine without root credentials. Running it through ssh I get the following:

~/.obstacle/obstacletower.x86_64 
No protocol specified
Found path: /home/stelios/.obstacle/obstacletower.x86_64
No protocol specified

Broken linux binary

This is on ubuntu 19.04, new installation. The character doesn't move and everything is broken.

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="19.04 (Disco Dingo)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 19.04"
VERSION_ID="19.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=disco
UBUNTU_CODENAME=disco


$nvidia-smi
Wed Sep  4 08:53:58 2019       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.56       Driver Version: 418.56       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:09:00.0  On |                  N/A |
| 24%   53C    P0    65W / 275W |    258MiB / 11175MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

Fireballs persist through floor resets

Going to a high level floor and losing or going up/down a level doesn't 'destroy' the fireball objects.

Steps to reproduce:

Start the environment.
Go to high level eg. 99.
Lose/reset the game.

On the left, the fireball objects persist through floors.

Chasing / fireball thrower bot walks through walls

Steps to reproduce:

Set seed to 100
Go to floor 98
Engage the bot that walks towards the agent
Run behind the wall in the first room so that the bot can't see the agent
Wait

The bot walks and shoots balls through the wall.

Upgrade to ml-agents 0.9.3

Is there a particular reason for supporting ml-agents v0.8.1?
Otherwise I'd just open up a PR for migrating to v0.9.3.

NullReferenceException in Procedural with dev tools

Using the procedural object in unity.

Steps to reproduce:

Pull procedural to unity editor
Press play
Click ToggleTimer
Press W once
Press space once

What caused the exception:

The issue is twofold;

W, S and Space interact with the dev tool buttons. W and S move up and down the selection, space causes a click
For some reason floorBuilder is pointing to a null object.
Note: Clicking on the buttons works just fine.

Stackframe:

NullReferenceException: Object reference not set to an instance of an object
UIController.GoToPrevFloor () (at Assets/ObstacleTower/Scripts/UI/UIController.cs:59)
UnityEngine.Events.InvokableCall.Invoke () (at C:/buildslave/unity/build/Runtime/Export/UnityEvent/UnityEvent.cs:166)
UnityEngine.Events.UnityEvent.Invoke () (at C:/buildslave/unity/build/Runtime/Export/UnityEvent/UnityEvent/UnityEvent_0.cs:58)
UnityEngine.UI.Button.Press () (at C:/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/UI/Core/Button.cs:66)
UnityEngine.UI.Button.OnSubmit (UnityEngine.EventSystems.BaseEventData eventData) (at C:/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/UI/Core/Button.cs:144)
UnityEngine.EventSystems.ExecuteEvents.Execute (UnityEngine.EventSystems.ISubmitHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at C:/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/EventSystem/ExecuteEvents.cs:127)
UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] functor) (at C:/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/EventSystem/ExecuteEvents.cs:261)
UnityEngine.EventSystems.EventSystem:Update()

Door does not render properly in Future Theme

Windows 10, Unity v2019.4.1f1

Shader: FT_GL_ForceField
Material: FT_ME_DoorRegular

Demonstrations in realtime mode can't retrace the path

Hi, I recorded some demonstrations from the Unity Editor. If I try to replay the demos from python in 'realtime' the path fails to retrace the demo.

Realtime mode renders only a black screen

Hi @awjuliani
if realtime_mode is set to True, the environment is not being rendered. Instead the window of OT stays black.
On the Unity side, ObstacleTowerAgent.cs:43 SetTraining() is awlays called, which disables the player's camera.
So there needs to be some functionality from the Python side to call SetInference().

edit: I don't know if the missing info can be retrieved directly from the ml-agents API. An alternative might be to add another reset parameter or to implement a custom side channel. What's your opinion?

unity-technologies / obstacle-tower-source Goto Github PK

obstacle-tower-source's Introduction

Obstacle Tower (Source code)

Reference paper

Training agents with a pre-built environment

Requirements

Opening the project

Understanding and extending the project

Obstacle Tower Challenge

obstacle-tower-source's People

Contributors

Stargazers

Watchers

Forkers

obstacle-tower-source's Issues

Steps to reproduce:

What caused the exception:

Stackframe:

Recommend Projects

Recommend Topics

Recommend Org