goxtool.py

goxtool.py is a trading client for the MtGox Bitcoin currency exchange. It allows you to watch live market data, to send or cancel orders or to run a fully automated trading bot. It is designed to work in the Linux console and it also natively works on Mac OS-X (Windows users see here), its written in Python and is using the (n)curses library to draw its user interface (it does not have a GTK or Qt interface), all it needs to run is a terminal (either locally or remote through ssh or putty).

This program is released under the GNU General Public License, Version 3.
The github project page with source code and issue tracker is here: https://github.com/prof7bit/goxtool
Donations are welcome: 1C8aDabADaYvTKvCAG1htqYcEgpAhkeYoW

When goxtool is running it looks like this:
goxtool.py screenshot of terminal running goxtool

How to install

Open a terminal in an empty folder or in the folder you usually use to clone repositories and clone the master branch:

git clone git://github.com/prof7bit/goxtool.git

This will create a folder named goxtool containing all the needed files. Thats all, now it is installed and ready to use. You can now already watch live market data (without any trading functions being enabled), you can later add a MtGox API-key to it to have full access to your account but for now just proceed to the next step, start it without an account, just to make sure everything works.

How to start it

change into to the goxtool folder that was created in the previous step and start goxtool.py:

cd goxtool
 ./goxtool.py

Keyboard commands (only the ones useful in view-only mode, without Mt.Gox account):

q quit
l (lower case "L") reload the strategy module (see advanced usage)
D (shift + d) switch to depth chart view
H (shift + h) switch to candlestick history chart view
S (shift + s) toggle summing up the volume of order book levels on/off
T (shift + t) toggle summing up the volume in the depth chart on/off
- order book zoom out (increase group size)
+ order book zoom in (decrease group size)
, depth chart zoom out (increase group size)
. depth chart zoom in (decrease group size)

(There will be even more commands once you connect it to your Mt.Gox account)

There is also a goxtool.ini file, it will be created on the first start. In the .ini file there are some parameters you can change, for example the currency you want to trade BTC against or some parameters regarding the network protocol. Some of the .ini settings can be overridden by command line options (use the --help option to see a list). The default protocol is websocket, the alternative would be socketio (and also recently added pubnub), goxtool implements all three protocols, the websocket server is currently more reliable. There is also an option to use the http API for trading commands instead sending them over the stream connection.

socketio, websocket or pubnub? Which one is worse?

The three options are socketio, websocket or pubnub, the .ini setting for this is use_plain_old_websocket to select between socketio or websocket, there is not yet an ini setting for pubnub, this will change in the near future. The protocol can also be forced on the command line and will take precedence over whatever is set in the ini.

To force it connecting with socketio:

./goxtool.py --protocol=socketio

To force it connecting to the websocket server:

./goxtool.py --protocol=websocket

To force it connecting to the pubnub server:

./goxtool.py --protocol=pubnub

It has turned out that websocket is currently the most reliable protocol, although the recently added new pubnub protocol might eventually take its place (it still has some problems occasionally, pubnub is conceptually inferior to websocket and producing twice as much network load and the MtGox pubnub also does not allow sending trading commands over the stream connection, so using pubnub will also automatically enable the http API for trading commands)

If you experience a high lag between sending an order and the ack (the op:result) not appearing immediately which is happening during times of really high volume (only websocket and socketio are affected) or mysteriously lost (ignored) trading commands then you should consider also adding --use-http to make it send trading commands via http. Http makes it slightly slower to execute many trades in a row but has been much more reliable when mtgox was under heavy load or ddos or similar. If you don't need to send many orders very fast then this option won't hurt. --use-http can be combined with either socketio or websocket. It is always automatically activated when using pubnub. You can also make it the default in the ini file.

This is what I am currently using and I recommend it:

./goxtool.py --protocol=websocket --use-http

When websocket is down due to ddos then I recommend the following:

./goxtool.py --protocol=pubnub

How to connect it to your MtGox account

Do the following:

./goxtool.py --add-secret

This will now ask you for key, secret and password. Key and secret belong to a shared secret that is created by MtGox to authenticate your trading software against their API. You can request as many keys from MtGox as you need, every application you connect to your MtGox account should have its own key, you can also at any time delete the keys again that you no longer need.

Now you need a Key/Secret pair for goxtool. Open your web browser, log in to your MtGox account, click on "Security Center", click on "Advanced API Key Creation", choose a name for your key (every key will have a name so you can later easily tell them apart), make sure you check at least the boxes "Get Info" and "Trade" (this is what the application will be allowed to do with this key) and then finally click on "Create Key".

Now MtGox will create 2 strings of cryptic numbers and letters, the "API-Key" and the "Secret". Now copy&paste the API-Key into the terminal where it asked you for "Key", press enter, now it will ask you for "Secret", copy&paste the Secret into the terminal, press enter and now it will ask you for a passphrase. It is important to understand what is going on here. The Key/Secret from above must under no circumstances ever come into the wrong hands, therefore goxtool won't just store them in the .ini file, goxtool will encrypt it and thats what the passphrase is needed for. Choose a secure passphrase (it will ask you twice to make sure there is no typo), you will also not see anything in the console (not even "*") while typing, this is not a bug, this is intentional. Choose a strong passphrase, type it into the terminal, press enter, repeat the passphrase, press enter again and now it will tell you that it has been encrypted and saved to the .ini file and exit.

Now start goxtool again:

./goxtool.py
enter passphrase for secret:

From now on every time you start goxtool it will ask you for the passphrase in order to be able to decrypt and use the secret. Enter your passphrase, press enter. Now goxtool will start and you will notice that now it is showing your account balance at the top of the window. Now all trading functions are enabled.

Keyboard commands for trading:

F4 : New buy order
F5 : New sell order
F6 : View orders / cancel order(s)

In the cancel dialog you can move up/down with the arrow keys, use INS to select/unselect orders (you can select multiple orders and cancel them all at once) or if you just quickly want to cancel only one order just highlight to the order and hit F8. It behaves a little bit like deleting files in midnight commander.

When entering a new order you can move between the fields with up/down keys or move to the next field with tab or enter (but only if you entered a valid number into the previous field, decimal separator is . (not comma, even on European computers), send the order with enter.

All dialogs can be closed with F10 or ESC.

Strategy module, strategy.py

You can write your own trading bots. There is a file named strategy.py, it contains a class Strategy() which constitutes a trading bot that by default does nothing (its an empty skeleton). It has event methods (slots) connected to signals that will be fired when certain events occur. From within these methods you can then do arbitrary stuff (peek around in gox.orderbook to see where bids and asks are located, call gox.buy(), gox.sell() or gox.cancel() methods to build a fully automated trading bot or you can use the key press slot (it will be called on all letter keys except l and q) to build a semi-automatic bot that reacts to key presses or to influence parameters of your bot or anything else you can imagine. Examples of simple bots will soon follow.

If you decide to make serious use of this then please create a new python file for your strategy. either make a copy of the default strategy.py skeleton or make a module that imports strategy and has a class Strategy(strategy.Strategy), give this module file a different name and leave strategy.py alone so it won't collide with upstream changes you pull from github. By default goxtool will load strategy.py but you can start it with the --strategy command line option to specify your own strategy module or a comma separated list of many modules:

./goxtool --strategy=mybot.py,myotherbot.py

You can even edit the strategy while goxtool is running and then reload it at runtime (this can be very useful), just press the l key (lowercase L) and it will do the following things:

emit signal _strategy_unload, this will call slot_before_unload()
free the currently running instance of Strategy() (your __del__() method should be called)
re-import the changed module file
create a new instance of Strategy() and call your__init__() again.

You should persist the state of your bot (if needed) in slot_before_unload() and reload it in __init__(). Leave the __del__() method alone, its only there to print a log message to debug proper unloading!

Please make sure that you can see the debug output from the __del__() method in the log when the strategy is reloading, you must be sure its able to free and garbage collect your strategy! If you instantiate any circular references, even something as innocent as a double linked list or even just an object holding a reference to the strategy then this will effectively keep python from being able to garbage-collect it and hold it in memory indefinitely (and keep sending it signals!).

Use the slot_before_unload() method to del everything in your strategy that might hold any circular references. You can check that it works if you see the debug output of __del__() in the log scrolling by when you press l to reload it, the fact that __del__() was called is proof that it was properly garbage-collected.

Trading functions do NOT block, this means they also won't return the MtGox order ID, you need to find your own way of remembering which orders you have sent already. A few moments (seconds or minutes) after you have sent them they will be acked by MtGox and it will fire orderbook.signal_changed()and when this happens you will find it in the gox.orderbook.owns list and it will have an official order ID. I know this is not optimal (because this part of the code is not yet complete, eventually there will be dedicated signals to notify your bot about the results of trading commands) and also this document is not yet a complete documentation. If you really want to dive into this: use the source, Luke.

How to keep it up to date

Occasionally I will commit bugfixes, improvements, etc. To update your copy of goxtool (assuming you previously installed it with git clone and not by just downloading a zip file) do the following:

git pull

and if that complains because of local uncommitted changes because you edited the strategy.py module or did other changes to the code then try this:

git stash
git pull
git stash pop

Of course you could have also have followed my previous advise to not do anything other than simple throw-away experiments in strategy.py so you can always git reset --hard if everything else fails and use a separate file (and probably even separate git branches) for serious bot development but this is outside the scope of this document, there exist specialized howtos for git and github elsewhere.

(TODO: finish writing this howto)