How To Use Open Ai Gym Classic Control Renderer

OpenAI Gym

OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. This is the gym open up-source library, which gives you access to a standardized gear up of environments.

See What's New section below

gym makes no assumptions almost the structure of your amanuensis, and is uniform with any numerical ciphering library, such as TensorFlow or Theano. You tin use it from Python code, and soon from other languages.

If you're not sure where to outset, nosotros recommend beginning with the docs on our site. See besides the FAQ.

A whitepaper for OpenAI Gym is available at http://arxiv.org/abs/1606.01540, and here's a BibTeX entry that you can utilise to cite it in a publication:

@misc{1606.01540,   Author = {Greg Brockman and Vicki Cheung and Ludwig Pettersson and Jonas Schneider and John Schulman and Jie Tang and Wojciech Zaremba},   Title = {OpenAI Gym},   Twelvemonth = {2016},   Eprint = {arXiv:1606.01540}, }

Contents of this document

Basics
Installation
- Installing everything
- Supported systems
- Pip version
- Rendering on a server
- Installing dependencies for specific environments
Environments
- Algorithmic
- Atari
- Box2d
- Classic command
- MuJoCo
- Robotics
- Toy text
Examples
Testing
What's new

Nuts

There are two bones concepts in reinforcement learning: the surround (namely, the outside world) and the agent (namely, the algorithm yous are writing). The agent sends actions to the environment, and the surroundings replies with observations and rewards (that is, a score).

The core gym interface is Env, which is the unified environment interface. There is no interface for agents; that part is left to you. The following are the Env methods you should know:

reset(self): Reset the environment'southward state. Returns observation.
footstep(self, action): Step the surround by i timestep. Returns observation, reward, done, info.
render(self, mode='human being', shut=False): Render one frame of the environment. The default mode will practise something human being friendly, such as pop up a window. Passing the close flag signals the renderer to close whatsoever such windows.

Installation

You can perform a minimal install of gym with:

git clone https://github.com/openai/gym.git              cd              gym pip install -eastward              .

If you prefer, you can exercise a minimal install of the packaged version directly from PyPI:

You'll be able to run a few environments correct away:

algorithmic
toy_text
classic_control (you'll demand pyglet to render though)

We recommend playing with those environments at beginning, so later installing the dependencies for the remaining environments.

Installing everything

To install the full set of environments, you'll need to have some organization packages installed. Nosotros'll build out the list here over time; please permit us know what you lot end up installing on your platform. Also, take a look at the docker files (exam.dockerfile.20.xx) to see the composition of our CI-tested images.

On OSX:

mash install cmake boost boost-python sdl2 swig wget

On Ubuntu 14.04 (non-mujoco only):

apt-get install libjpeg-dev cmake swig python-pyglet python3-opengl libboost-all-dev \         libsdl2-2.0.0 libsdl2-dev libglu1-mesa libglu1-mesa-dev libgles2-mesa-dev \         freeglut3 xfvb libav-tools

On Ubuntu xvi.04:

apt-go install -y python-pyglet python3-opengl zlib1g-dev libjpeg-dev patchelf \         cmake swig libboost-all-dev libsdl2-dev libosmesa6-dev xvfb ffmpeg

On Ubuntu 18.04:

apt install -y python3-dev zlib1g-dev libjpeg-dev cmake swig python-pyglet python3-opengl libboost-all-dev libsdl2-dev \     libosmesa6-dev patchelf ffmpeg xvfb

MuJoCo has a proprietary dependency we can't gear up for you. Follow the instructions in the mujoco-py package for help.

Once you're ready to install everything, run pip install -eastward '.[all]' (or pip install 'gym[all]').

Supported systems

We currently support Linux and Bone 10 running Python ii.7 or 3.5. Some users on OSX + Python3 may need to run

brew install boost-python --with-python3

If you desire to admission Gym from languages other than python, we take express support for non-python frameworks, such equally lua/Torch, using the OpenAI Gym HTTP API.

Pip version

To run pip install -e '.[all]', yous'll need a semi-recent pip. Please make sure your pip is at least at version 1.v.0. You can upgrade using the following: pip install --ignore-installed pip. Alternatively, you tin can open setup.py and install the dependencies by paw.

Rendering on a server

If you lot're trying to return video on a server, you'll need to connect a fake display. The easiest way to do this is past running under xvfb-run (on Ubuntu, install the xvfb parcel):

xvfb-run -due south                              "-screen 0 1400x900x24"                            fustigate

Installing dependencies for specific environments

If you'd similar to install the dependencies for only specific environments, come across setup.py. We maintain the lists of dependencies on a per-environment grouping basis.

Environments

The lawmaking for each environment grouping is housed in its own subdirectory gym/envs. The specification of each task is in gym/envs/__init__.py. Information technology's worth browsing through both.

Algorithmic

These are a variety of algorithmic tasks, such every bit learning to copy a sequence.

              import              gym              env              =              gym.make('Copy-v0')              env.reset()              env.render()

Atari

The Atari environments are a variety of Atari video games. If you lot didn't do the total install, you can install dependencies via pip install -e '.[atari]' (you'll need cmake installed) so go started as follow:

              import              gym              env              =              gym.brand('SpaceInvaders-v0')              env.reset()              env.return()

This will install atari-py, which automatically compiles the Arcade Learning Environment. This can take quite a while (a few minutes on a decent laptop), so but be prepared.

Box2d

Box2d is a 2D physics engine. You can install information technology via pip install -e '.[box2d]' and and so get started as follow:

              import              gym              env              =              gym.brand('LunarLander-v2')              env.reset()              env.render()

Archetype command

These are a variety of classic control tasks, which would appear in a typical reinforcement learning textbook. If you didn't do the full install, you lot will demand to run pip install -eastward '.[classic_control]' to enable rendering. You can go started with them via:

              import              gym              env              =              gym.make('CartPole-v0')              env.reset()              env.return()

MuJoCo

MuJoCo is a physics engine which can practice very detailed efficient simulations with contacts. It's non open-source, so you'll accept to follow the instructions in mujoco-py to set up it up. Y'all'll have to besides run pip install -e '.[mujoco]' if you didn't exercise the total install.

              import              gym              env              =              gym.make('Humanoid-v2')              env.reset()              env.render()

Robotics

MuJoCo is a physics engine which can do very detailed efficient simulations with contacts and nosotros utilize it for all robotics environments. It's not open-source, and so you'll have to follow the instructions in mujoco-py to set it upwards. Y'all'll have to also run pip install -e '.[robotics]' if you didn't do the full install.

              import              gym              env              =              gym.make('HandManipulateBlock-v0')              env.reset()              env.render()

You lot can also detect additional details in the accompanying technical report and web log post. If you apply these environments, you tin cite them as follows:

@misc{1802.09464,   Writer = {Matthias Plappert and Marcin Andrychowicz and Alex Ray and Bob McGrew and Bowen Baker and Glenn Powell and Jonas Schneider and Josh Tobin and Maciek Chociej and Peter Welinder and Vikash Kumar and Wojciech Zaremba},   Championship = {Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Asking for Enquiry},   Year = {2018},   Eprint = {arXiv:1802.09464}, }

Toy text

Toy environments which are text-based. There'south no extra dependency to install, then to get started, you lot can just exercise:

              import              gym              env              =              gym.make('FrozenLake-v0')              env.reset()              env.render()

Examples

See the examples directory.

Run examples/agents/random_agent.py to run an simple random agent.
Run examples/agents/cem.py to run an actual learning agent (using the cross-entropy method).
Run examples/scripts/list_envs to generate a listing of all environments.

Testing

Nosotros are using pytest for tests. You can run them via:

What's new

2018-02-28: Release of a ready of new robotics environments.
2018-01-25: Made some artful improvements and removed unmaintained parts of gym. This may seem like a downgrade in functionality, but it is really a long-needed cleanup in preparation for some great new things that will be released in the side by side month.
- Now your Env and Wrapper subclasses should define step, reset, return, close, seed rather than underscored method names.
- Removed the board_game, debugging, safety, parameter_tuning environments since they're non existence maintained past us at OpenAI. We encourage authors and users to create new repositories for these environments.
- Changed MultiDiscrete activeness space to range from [0, ..., n-1] rather than [a, ..., b-1].
- No more render(close=True), use env-specific methods to close the rendering.
- Removed scoreboard directory, since site doesn't be anymore.
- Moved gym/monitoring to gym/wrappers/monitoring
- Add together dtype to Space.
- Not using python'southward born module anymore, using gym.logger
2018-01-24: All continuous command environments now employ mujoco_py >= one.50. Versions have been updated accordingly to -v2, e.g. HalfCheetah-v2. Performance should be similar (see openai/gym#834) but there are probable some differences due to changes in MuJoCo.
2017-06-16: Make env.spec into a property to prepare a issues that occurs when you lot effort to print out an unregistered Env.
2017-05-13: BACKWARDS INCOMPATIBILITY: The Atari environments are at present at v4. To keep using the onetime v3 environments, keep gym <= 0.8.2 and atari-py <= 0.0.21. Annotation that the v4 environments will not give identical results to existing v3 results, although differences are small-scale. The v4 environments incorporate the latest Arcade Learning Environment (ALE), including several ROM fixes, and at present handle loading and saving of the emulator country. While seeds even so ensure determinism, the effect of whatsoever given seed is not preserved across this upgrade because the random number generator in ALE has changed. The *NoFrameSkip-v4 environments should be considered the canonical Atari environments from now on.
2017-03-05: BACKWARDS INCOMPATIBILITY: The configure method has been removed from Env. configure was not used past gym, merely was used by some dependent libraries including universe. These libraries will migrate abroad from the configure method by using wrappers instead. This modify is on main and will be released with 0.8.0.
2016-12-27: BACKWARDS INCOMPATIBILITY: The gym monitor is at present a wrapper. Rather than starting monitoring as env.monitor.start(directory), envs are now wrapped as follows: env = wrappers.Monitor(env, directory). This modify is on primary and volition be released with 0.7.0.
2016-11-ane: Several experimental changes to how a running monitor interacts with environments. The monitor volition now heighten an fault if reset() is called when the env has not returned washed=True. The monitor volition only record consummate episodes where washed=True. Finally, the monitor no longer calls seed() on the underlying env, nor does it record or upload seed information.
2016-10-31: We're experimentally expanding the environs ID format to include an optional username.
2016-09-21: Switch the Gym automated logger setup to configure the root logger rather than just the 'gym' logger.
2016-08-17: Calling close on an env will also shut the monitor and any rendering windows.
2016-08-17: The monitor will no longer write manifest files in existent-time, unless write_upon_reset=Truthful is passed.
2016-05-28: For controlled reproducibility, envs now support seeding (cf #91 and #135). The monitor records which seeds are used. We will presently add seed information to the display on the scoreboard.