OANDA Trading Environment: getting started

Using the OANDA REST-API to produce records in a configurable way that suits your needs for providing data for your trading applications and enable you to auto-trade via the REST-API.

OANDA Trading Environment: the picture

alt text

The OANDAd daemon of the OANDA Trading Environment serves as a local quoteserver generating OHLC-records / candle records for different timeframes. It uses the OANDA REST-API to get streaming quotes by using the oandapy API wrapper.

Client applications can subscribe for quotes via a ZMQ subscription. The OANDAd will provide these applications continuously with candle-records by publishing these records.

The daemon provides the concept of plugins. This allows pieces of Python code te be run as integrated code when the daemon has produced a record. Plugins are used to publish records, store records in flat file stucture or store records in a database. See plugins for details.

Streaming Candles

Main part of the OANDA Trading Environment is the OANDAd daemon that parses the streaming quotes from the OANDA-API in configurable timeframes, by 1 minute, 5 minutes, 15 minutes etc. This makes it produce streaming candles. This is mostly useful for the shorter timeframes. The larger timeframes can be requested using the API.

Candle data:

{"data": {"instrument": "EUR_JPY",
                "granularity" : "M1",
                "start": "2015-09-02 15:36:00"
                "end": "2015-09-02 15:37:00",
                "completed": true,
                "data": {"high": 134.967, 
                         "open": 134.962,
                         "last": 134.9565,
                          "low": 134.9475,
                       "volume": 19
                 },
               }
       }

Streaming data can be controlled by the fabricate setting in the streamer config section of OANDAd.cfg. See controlling record types for details.

Dataformat

The dataformat as shown can be easily altered. With a few lines of code in the plugin you are free to change the format and add attributes before you publish the record.

Plugins: actions on candlerecords

When a timeframe is completed it can be handled by one or more plugins. Plugins have a configfile based on the name of the plugin-file, but in lowercase. The plugins can be found under:

etc/OANDA/plugins

and the plugin configs under

etc/OANDA/config/plugins

If you want to use a plugin then you need to enabled it in the config file, see the OANDAd.cfg:

# etc/OANDA/config/OANDAd.cfg
...
plugins:
  ...
  enabled:
    - plainfile
    - pubsub

The environment comes with a few plugins:

Publish/Subscribe - plugin

This plugin can be configured to publish the candlerecord using a publisher/subscriber mechanism. This is achieved by using the 0MQ library and the python binding for it: pyzmq, see http://zeromq.org for details. Trading applications can easily subscribe to receive the candle data with just a few lines of code, see pubsub for details.

This plugin is enabled by default and publishes on 127.0.0.1, localhost. If you want to subscribe with clients from other hosts, reconfigure the pubsub plugin by changing the pubsub.cfg file and change the IP-address to the address of your OTE-host.

Plainfile - plugin

This plugin can be configured to write candle records to a flatfile in a directory structure, which will result in a structure like below:

Example:

/tmp/oandadb
     |-- BCO_USD
     |   |-- M1
     |   |   `-- cache
     |   |-- M15
     |   |   `-- cache
     |   `-- M5
     |       `-- cache
     |-- DE30_EUR
     |   |-- M1
     |   |   `-- cache
     |   |-- M15
     |   |   `-- cache
     |   `-- M5
     |       `-- cache
     |-- EUR_CHF
     |   |-- M1
     |   |   `-- cache
     |   |-- M15
     |   |   `-- cache
     |   `-- M5
     |       `-- cache
     |-- EUR_GBP
     |   |-- M1
     |   |   `-- cache
     |   |-- M15
     |   |   `-- cache
     |   `-- M5
     |       `-- cache
     |-- EUR_JPY
     |   |-- M1
     |   |   `-- cache
     |   |-- M15
     |   |   `-- cache
     |   `-- M5
     |       `-- cache
     |-- EUR_USD
         |-- M1
         |   `-- cache
         |-- M15
         |   `-- cache
         `-- M5
             `-- cache

MySQL - plugin

The MySQL plugin can be configured to insert records into a database. This plugin is provided as an example, since it needs details that depend on your databasemodel.

Trading applications as ZMQ client

Write your trading applications by making use of the ZMQ pub/sub facility of OANDAd. This way you can completely isolate your trading code from the OANDAd daemon. It also enables you to write applications in a different language then Python, the only requirement is that is must support ZMQ. See, example for a ZMQ subscription example.

Security

The environment makes use of a token that gives access to sensitive information.

Please pay attention to where you install this software. Never use this on a system that is not owned and controlled by yourself.

Make sure to secure your system as much as possible:

  • restrict network access
  • make no use of, or limit other network services (NFS, SAMBA, printerserver etc.)
  • limit or deny user access, preferrable only you
  • use encryption to access the system (SSH)
  • how is physical access ?

Requirements

  • Python 2.7.3
  • oandapy
  • pyyaml > 3.10
  • daemonocle >= 0.8
  • pyzmq >= 14.7

Installation

Currently there are two ways to install: pip and git.

Using Git

user $ mkdir <somewhere>
user $ cd <somewhere>
user $ mkdir OANDA
user $ virtualenv OANDA_env

# optionally use virtualenv --system-site-packages to use the standard
# available packages for the python modules available on your distribution:
#   _pyyaml_, _pyzmq_. Check for the packages on the distribution.

user $ . ./OANDA_env/bin/activate
user $ git clone https://github.com/hootnot/oanda-trading-environment.git
user $ cd oanda-trading-environment
user $ python setup.py install

OANDA has not (yet) made the oandapy module pip installable, either from git or pypi.python.org.

To install the latest oandapy you can use this fork to install it with pip:

user $ pip install git+https://github.com/hootnot/oandapy

user $ pip list | grep oanda
oanda-trading-environment (0.0.1)
oandapy (0.1)

Using pip and the latest pypi-package

user $ mkdir <somewhere>
user $ cd <somewhere>
user $ mkdir OANDA
user $ virtualenv [--system-site-packages] OANDA_env

# optionally use virtualenv --system-site-packages to use the standard
# available packages for the python modules available on your distribution:
#   _pyyaml_, _pyzmq_. Check for the packages on the distribution.

user $ . ./OANDA_env/bin/activate
user $ pip install oanda-trading-environment
user $ pip install git+https://github.com/hootnot/oandapy

Using a systemwide install:

user $ sudo pip install oanda-trading-environment
user $ sudo pip install git+https://github.com/hootnot/oandapy

Configuring OANDAd

By default OANDAd is set with a number of instruments to follow for different granularities.

For the initial configuaration you only need to enter the token and account_id.

Edit the config file and replace the placeholders:

# etc/OANDA/config/OANDAd.cfg
...
access_token: _token_from_oanda_here
account_id: _accountid_here_
...

Controlling OANDAd

OANDAd is built using daemonocle. The ‘start’, ‘status’ and ‘stop’ commands are implemented. The daemon forks itself and the child will process the stream. When there are issues, TIME-OUT for instance, the child will exit and a new child will be spawned.

Starting the OANDAd

user $ OANDAd start
Starting OANDAd ... OK

The daemon will process streaming quotes now and process the timeframes as configured. Timeframes are currently based on the midprice of bid/ask. During processing candle records will be produced. The way timeframes will be processed to produce candle records can be controlled.

Status of the OANDAd

user $ OANDAd status
OANDAd -- pid: 51931, status: sleeping, uptime: 0m, %cpu: 0.0, %mem: 1.8

Stopping the OANDAd

user $ OANDAd stop
Stopping OANDAd ... OK

Controlling recordtypes

Records can be produced in different ways by changing the fabricate setting of the streamer:

streamer:
  ...
  fabricate: atEndOfTimeFrame|dancingBear|dancingBearHighLowExtreme

atEndOfTimeFrame

The setting generating least number of records is: atEndOfTimeFrame. A record is only created when the timeperiod of the timeframe has passed.

dancingBear

This setting generates a record for each tick it processes. Mostly a lot and in volatile markets a lot more.

dancingBearHighLowExtreme

This setting generates also dancingBear records but only when a new high or a new low is set for the timeframe. When a market is volatile you still get a lot of records because it is likely that a lot of ticks will lead to new highs or lows within the timeframe.

But in less volatile markets you will have a lot of ticks that will stay between the high and low set at a certain moment. Therefore this will generate a lot less records but still give you the relevant details at the moment they occur.

If you want dancingBear records consider this option instead of the dancingBear option.

Logging

The ticks received from the stream are written to a logfile:

<installation_root>/var/log/OANDA/streamdata.<date>

The OANDAd itself logs to:

<installation_root>/var/log/OANDA/OANDAd.log

Loglevel and the streamdata logfile extension is configurable. Check the OANDAd.cfg for details.