diff --git a/README.md b/README.md index 8d632c35f..99473749e 100644 --- a/README.md +++ b/README.md @@ -1,51 +1,32 @@ -# Important note +### Gekko (*origin of the name*) -You are looking at the brand new and completetly different version of Gekko. We've tested it for quite a while though it might be possible that you encounter bugs. If you encounter a bug: check out in [the issues](https://github.com/askmike/gekko/issues/) if we are aware of it and if not create a new one :) +#### 1987 movie "Wall Street" +> *The most valuable commodity I know of is information.* +> -- Gordon Gekko -# Gekko [![Build Status](https://travis-ci.org/askmike/gekko.png)](https://travis-ci.org/askmike/gekko) +## Gekko (*this tool / bot*) -![Gordon Gekko](http://mikevanrossum.nl/static/gekko.jpg) +This tool is based on [REST APIs](http://en.wikipedia.org/wiki/Representational_state_transfer), and is written the [node.js](https://github.com/joyent/node) dialect of javascript, which itself based on v8, Google's open source JavaScript engine. +

 
 

+**WARNING:** *Use Gekko at you own risk.* -*The most valuable commodity I know of is information.* +--- --Gordon Gekko +### Automated reinvestment -Gekko is a Bitcoin trading bot and backtesting platform that connects to popular Bitcoin exchanges. It is written in javascript and runs on [nodejs](http://nodejs.org). +Gekko comes with a few [technical analysis (TA)](http://en.wikipedia.org/wiki/Technical_analysis) methods which implement a single indicator (*DEMA, MACD, RSI, and PPO*). The parameters of these indicators are all configurable and changing them changes the outcome drastically. Gekko's interface allows you to add your own trading methods. Read more about that in the [documentation](https://github.com/kuzetsa/gekko/blob/master/docs/internals/trading_methods.md). -This is the open source do-it-yourself version, we are planning on running hosted Gekkos in the cloud which does not require you to download and install anything, configure any textfiles or deal with the commandline. If you are looking for such a solution, sign up at [Wizbit](http://wizb.it) and we'll let you know once it's out! +**Note:** *much of this information is outdated, and/or not relevant to reinvestment-only mode.* -*Use Gekko at you own risk.* +### Supported exchanges -## Main features +Gekko reinvests cloud mining income on the following platforms: -* Trading platform: - * Paper trading - * Live trading (trade bot) - * ~~Backtester~~ -* Market interface: - * Emit market events - * Basic IRC Bot +- CEX.io (*Currently only BTC, planned support for NMC reinvestment.*) -## Automated Trading platform +### Plugin interface -Gekko can watch the realtime markets. You can apply automated trading methods to realtime data coming in to do live or simulated trading (automated trading or paper trading). Gekko also stores the market data it sees so you can run the trading methods with simulate trades on a set of historical data to see whether they would have been profitable during that time (backtesting). - -Gekko, as well as the current bitcoin exchanges, are not built for HFT or anything related to being the fastest. The trading methods Gekko can do are based on indicators used by human day traders. The result is that Gekko does not look at data below the one minute timescale and (depending on configuration) and will normally not trade more than a couple of times per week (also depending on configuration). - -**So Gekko is not:** - -- A trading platform for human day traders with a GUI and charts. -- A High frequency trading bot designed to operate on < minute resolution. -- A fully automated trading bot that you turn on and will generate profit without you having to do anything. -- An exchange. - -**Automated trading** - -Gekko comes with a couple of trading methods that implement a single indicator (DEMA, MACD, RSI and PPO). The parameters of these indicators are all configurable and changing them changes the outcome drastically. Additionally Gekko also provides an easy way to write your own trading methods in javascript. Read more about that in the [documentation](https://github.com/askmike/gekko/blob/master/docs/internals/trading_methods.md). - -## Market interface - -Gekko also has a plugin system that can do certain things whenever something happens or let Gekko communicate through more platforms. Gekko currently knows these plugins: +Gekko also has a plugin system that can do certain things whenever something happens or let Gekko communicate through more platforms. Gekko implements the following example plugins: - Campfire: Enables Gekko to talk on [Campfire](https://campfirenow.com/) and report latest market data and advice. - IRC bot: Enables Gekko to talk on IRC and report latest market data and advice. @@ -53,86 +34,93 @@ Gekko also has a plugin system that can do certain things whenever something hap - Profit Simulator (paper trader): Hold a fake portfolio and simulate trades based on advice. - Redis Beacon: Broadcast events propagating through Gekko on [Redis pub/sub](http://redis.io/topics/pubsub). -## Supported exchanges - -Gekko works on the following exchanges: - -- Mt. Gox -- Bitstamp -- CEX.io -- Kraken -- BTC-e -- ~~Cryptsy~~ (In the [pipeline](https://github.com/askmike/gekko/pull/200)) +### Installing Gekko -## Installing Gekko +Windows user? Here is a [step-by-step guide](https://github.com/kuzetsa/gekko/blob/master/docs/installing_gekko_on_windows.md) on how to get Gekko running on Windows. -Windows user? Here is a [step-by-step guide](https://github.com/askmike/gekko/blob/master/docs/installing_gekko_on_windows.md) on how to get Gekko running on Windows. +Gekko runs on [nodejs](http://nodejs.org/), once you have that installed you can either download all files in [a zip](https://github.com/kuzetsa/gekko/archive/master.zip) or clone the repository via git: -Gekko runs on [nodejs](http://nodejs.org/), once you have that installed you can either download all files in [a zip](https://github.com/askmike/gekko/archive/master.zip) or clone the repository via git: - - git clone git://github.com/askmike/gekko.git +``` + git clone git://github.com/kuzetsa/gekko.git cd gekko +``` You need to download Gekko's dependencies, which can easily be done with [npm](http://npmjs.org): +``` npm install +``` -## Configuring Gekko - -> Configuring Gekko consists of three parts: -> -> - Watching a realtime market -> - Automate trading advice -> - Enabling plugins +### Configuring Gekko / before you start -Read the [configuring Gekko documentation](https://github.com/askmike/gekko/tree/master/docs/Configuring_gekko.md) for a detailed explanation. +Read the [configuring Gekko documentation](https://github.com/kuzetsa/gekko/tree/master/docs/Configuring_gekko.md) for a detailed explanation. -## Running Gekko +### Running Gekko To run the bot you just have to start Gekko: +``` node gekko +``` -You can also run Gekko silently or use more complex features, for examples check out the [advanced features](https://github.com/askmike/gekko/tree/master/docs/Advanced_features.md). +You can also run Gekko silently or use more complex features, for examples check out the [advanced features](https://github.com/kuzetsa/gekko/tree/master/docs/Advanced_features.md). -## Updating Gekko +### Updating Gekko If you installed the bot via git you can easily fetch the latest updates by running: - git pull && npm update +``` + git pull + npm update +``` -## How does Gekko work? - -![Gekko 0.1.0 architecture](http://data.wizb.it/misc/gekko-0.1.0-architecture.jpg) +### How does Gekko work? If you want to contribute or are interested in how Gekko works: -- Read about [Gekko's overall architecture](https://github.com/askmike/gekko/tree/master/docs/internals/architecture.md). -- Read on how to add [a new exchange to Gekko](https://github.com/askmike/gekko/tree/master/docs/internals/exchanges.md). -- Read on how to [create your own plugin](https://github.com/askmike/gekko/tree/master/docs/internals/plugins.md). -- Implement [your own trading method](https://github.com/askmike/gekko/blob/master/docs/internals/trading_methods.md) and share it back. - -## TODO - -* Backtester -* More exchanges (bitfinex, btcchina) -* More indicators -* Webbased interface +- Read about [Gekko's overall architecture](https://github.com/kuzetsa/gekko/tree/master/docs/internals/architecture.md). +- Read on how to add [a new exchange to Gekko](https://github.com/kuzetsa/gekko/tree/master/docs/internals/exchanges.md). +- Read on how to [create your own plugin](https://github.com/kuzetsa/gekko/tree/master/docs/internals/plugins.md). +- Implement [your own trading method](https://github.com/kuzetsa/gekko/blob/master/docs/internals/trading_methods.md) and share it back. ## Credits * The title is inspired by [Bateman](https://github.com/fearofcode/bateman). * This project is inspired by the [GoxTradingBot](https://github.com/virtimus/GoxTradingBot/) Chrome plugin (though no code is taken from it). +* Original implementation of gekko was written by [Mike van Rossum](https://github.com/askmike/gekko) +* After 2014, June 25th, reinvestment-only fork maintained by [Sarah White AKA kuzetsa](https://github.com/kuzetsa/gekko) +* Credits for the reduced-lag EMA methods go to Patrick G. Mulloy for publishing
"Smoothing Data With Less Lag" in 1994 + +### License + +(*Legal rights and restrictions to copy and use this code*) + +#### [GNU Affero General Public License (AGPL) version 3](https://github.com/kuzetsa/gekko/blob/c325e25ba78fd9657dff275a4f901311755e7c67/LICENSE.md) + +Copyright (c) 2013-2014 Sarah White (*AKA* **kuzetsa**, *forked 2014 June 25th*) + + This file is part of gekko. + + gekko is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + gekko is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. -## Final + You should have received a copy of the GNU Affero General Public License + along with gekko. If not, see . -If Gekko helped you in any way, you can always leave me a tip at (BTC) 13r1jyivitShUiv9FJvjLH7Nh1ZZptumwW +--- -## License +> I, [Sarah White (kuzetsa)](https://github.com/kuzetsa/) have personally opted to use the "sublicensing" right under the permissive MIT license to sublicense my own work. [All of my own code and documentation commits](https://github.com/kuzetsa/gekko/commits?author=kuzetsa) in this project are under the [AGPLv3 license](https://github.com/kuzetsa/gekko/blob/c325e25ba78fd9657dff275a4f901311755e7c67/LICENSE.md) -The MIT License (MIT) +#### The MIT License (MIT) -Copyright (c) 2014 Mike van Rossum +Copyright (c) 2014 Mike van Rossum (*original author, initial code in gekkobot*) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal @@ -150,4 +138,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. \ No newline at end of file +THE SOFTWARE. diff --git a/bin/gekko_launch.sh b/bin/gekko_launch.sh deleted file mode 100755 index 442f61aab..000000000 --- a/bin/gekko_launch.sh +++ /dev/null @@ -1,18 +0,0 @@ -#navigate to gekko folder, change this if your location is different -cd ~/gekko - -# zip previous log files for this config -now="`date +%Y%m%d%H%M%S`" -current_log=log/gekko_log.$1.txt -archive_log=log/gekko_log.$1.$now.txt - -if [ -f $current_log ]; then - mv log/gekko_log.$1.txt $archive_log - zip -vu log/gekko_logs.zip $archive_log - - # remove raw text files now that they're zipped - rm $archive_log -fi - -# finally launch gekko and log output to log file as well as stdout -node gekko config="config/user/$1.js" 2>&1 | tee $current_log diff --git a/bin/gekko_launch_screen.sh b/bin/gekko_launch_screen.sh deleted file mode 100755 index f1e88249c..000000000 --- a/bin/gekko_launch_screen.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -screen -dmS gekko_$1 ~/gekko/bin/gekko_launch.sh $1 - diff --git a/bin/gekko_log_grab.sh b/bin/gekko_log_grab.sh deleted file mode 100755 index bad586ca2..000000000 --- a/bin/gekko_log_grab.sh +++ /dev/null @@ -1 +0,0 @@ -tail -f ~/gekko/log/gekko_log.$1.txt diff --git a/bin/gekko_screen_grab.sh b/bin/gekko_screen_grab.sh deleted file mode 100755 index 88ff923ca..000000000 --- a/bin/gekko_screen_grab.sh +++ /dev/null @@ -1 +0,0 @@ -screen -dR gekko_$1 diff --git a/config.js b/config.js index 1f53c50df..0387fe55e 100644 --- a/config.js +++ b/config.js @@ -1,5 +1,5 @@ -// Everything is explained here: -// https://github.com/askmike/gekko/blob/master/docs/Configuring_gekko.md +// Everything is explained here: +// https://github.com/kuzetsa/gekko/blob/master/docs/Configuring_gekko.md var config = {}; @@ -12,8 +12,9 @@ config.history = { // in what directory should Gekko store // and load historical data from? directory: './history/' -} -config.debug = true; // for additional logging / debugging +}; + +config.debug = false; // for additional logging / debugging // ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ // WATCHING A MARKET @@ -22,10 +23,10 @@ config.debug = true; // for additional logging / debugging // Monitor the live market config.watch = { enabled: true, - exchange: 'Bitstamp', // 'MtGox', 'BTCe', 'Bitstamp', 'cexio' or 'kraken' - currency: 'USD', - asset: 'BTC' -} + exchange: 'cexio', // 'MtGox', 'BTCe', 'Bitstamp', 'cexio' or 'kraken' + currency: 'BTC', + asset: 'GHS' +}; // ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ // CONFIGURING TRADING ADVICE @@ -33,10 +34,10 @@ config.watch = { config.tradingAdvisor = { enabled: true, - method: 'DEMA', - candleSize: 60, - historySize: 50 -} + method: 'ZERO', + candleSize: 1, + historySize: 2345 +}; // Exponential Moving Averages settings: config.DEMA = { @@ -56,16 +57,73 @@ config.DEMA = { config.MACD = { // EMA weight (α) // the higher the weight, the more smooth (and delayed) the line - short: 10, - long: 21, - signal: 9, + short: 53, + long: 109, + signal: 41, // the difference between the EMAs (to act as triggers) thresholds: { - down: -0.025, - up: 0.025, - // How many candle intervals should a trend persist - // before we consider it real? - persistence: 1 + down: -9999, + up: 0.00000001, + } +}; + +// x2MACD settings: +config.x2MACD = { + // EMA weight (α) + // the higher the weight, the more smooth (and delayed) the line + short: 53, + long: 109, + signal: 41, + // the difference between the EMAs (to act as triggers) + thresholds: { + down: -9999, + up: 0.00000001, + } +}; + +// nikiehihsa settings: +config.nikiehihsa = { + // EMA weight (α) + // the higher the weight, the more smooth (and delayed) the line + short: 53, + long: 109, + signal: 41, + // the difference between the EMAs (to act as triggers) + thresholds: { + down: -9999, + up: 0.00000001, + } +}; + +// x3nikiehihsa settings: +config.x3nikiehihsa = { + // EMA weight (α) + // the higher the weight, the more smooth (and delayed) the line + short: 53, + long: 109, + signal: 41, + // the difference between the EMAs (to act as triggers) + thresholds: { + down: -9999, + up: 0.00000001, + } +}; + +// ZERO settings: +config.ZERO = { + // EMA weight (α) + // the higher the weight, the more smooth (and delayed) the line + short: 257.608488, + long: 364.313417, + signal: 225.158074, + // how optimistic is the MACD extrapolation going to be? + crystalball: 0.00000173, + // how large is the stats window for sanity checking? + window: 2345, // SHOULD NOT be larger than your historySize!!! + // the difference between the EMAs (to act as triggers) + thresholds: { + down: -9999, + up: 0.00000042, } }; @@ -78,11 +136,8 @@ config.PPO = { signal: 9, // the difference between the EMAs (to act as triggers) thresholds: { - down: -0.025, - up: 0.025, - // How many candle intervals should a trend persist - // before we consider it real? - persistence: 2 + down: -9999, + up: 0.00000001, } }; @@ -112,9 +167,9 @@ config.custom = { // watched by config.watch config.trader = { enabled: false, - key: '', - secret: '', - username: '' // your username, only fill in when using bitstamp or cexio + key: 'see next line', + secret: 'some sort of hash goes here', + username: 'fakename change this', // your username, as required by cexio } config.adviceLogger = { @@ -123,7 +178,7 @@ config.adviceLogger = { // do you want Gekko to calculate the profit of its own advice? config.profitSimulator = { - enabled: true, + enabled: false, // report the profit in the currency or the asset? reportInCurrency: true, // start balance, on what the current balance is compared with @@ -133,11 +188,11 @@ config.profitSimulator = { currency: 100, }, // only want report after a sell? set to `false`. - verbose: false, + verbose: true, // how much fee in % does each trade cost? - fee: 0.6, + fee: 1, // how much slippage should Gekko assume per trade? - slippage: 0.05 + slippage: 0.5 } // want Gekko to send a mail on buy or sell advice? @@ -150,7 +205,7 @@ config.mailer = { // You don't have to set your password here, if you leave it blank we will ask it // when Gekko's starts. // - // NOTE: Gekko is an open source project < https://github.com/askmike/gekko >, + // NOTE: Gekko is an open source project < https://github.com/kuzetsa/gekko >, // make sure you looked at the code or trust the maintainer of this bot when you // fill in your email and password. // @@ -176,6 +231,25 @@ config.mailer = { tls: false // Use TLS if true } +config.mandrillMailer = { + enabled: false, + sendMailOnStart: true, + to: '', // to email + toName: 'Gekko user', + from: '', // from email + fromName: 'Gekko bot info', + apiKey: '', // Mandrill api key +} + +config.smsPlivo = { + enabled: false, + sendMailOnStart: true, + smsPrefix: 'GEKKO:', // always start SMS message with this + to: '', // your SMS number + from: '', // SMS number to send from provided by Plivo + authId: '', // your Plivo auth ID + authToken: '' // your Plivo auth token +} config.ircbot = { enabled: false, @@ -209,7 +283,8 @@ config.redisBeacon = { ] } -// not in a working state +// the web interface not currently supported, maintainer of the plugin quit +// (not in a working state) // read: https://github.com/askmike/gekko/issues/156 config.webserver = { enabled: false, @@ -238,7 +313,6 @@ config.backtest = { // it doesn't advice on itself, only set to true if you truly // understand this. // -// Not sure? Read this first: https://github.com/askmike/gekko/issues/201 config['I understand that Gekko only automates MY OWN trading strategies'] = false; -module.exports = config; \ No newline at end of file +module.exports = config; diff --git a/core/baseTradingMethod.js b/core/baseTradingMethod.js index b8a0cbe96..966f9dd6f 100644 --- a/core/baseTradingMethod.js +++ b/core/baseTradingMethod.js @@ -6,6 +6,22 @@ var log = require('../core/log.js'); var indicatorsPath = '../methods/indicators/'; var Indicators = { + ZERO: { + factory: require(indicatorsPath + 'ZERO'), + input: 'candle' + }, + x3nikiehihsa: { + factory: require(indicatorsPath + 'x3nikiehihsa'), + input: 'candle' + }, + nikiehihsa: { + factory: require(indicatorsPath + 'nikiehihsa'), + input: 'candle' + }, + x2MACD: { + factory: require(indicatorsPath + 'x2MACD'), + input: 'price' + }, MACD: { factory: require(indicatorsPath + 'MACD'), input: 'price' @@ -40,12 +56,14 @@ var Base = function() { // make sure we have all methods _.each(['init', 'check'], function(fn) { - if(!this[fn]) - util.die('No ' + fn + ' function in this trading method found.') + if(!this[fn]) { + util.die('No ' + fn + ' function in this trading method found.'); + } }, this); - if(!this.update) + if(!this.update) { this.update = function() {}; + } // let's run the implemented starting point this.init(); @@ -53,16 +71,18 @@ var Base = function() { // should be set up now, check some things // to make sure everything is implemented // correctly. - if(this.name) + if(this.name) { log.info('\t', 'Using trading method:', this.name); - else + } else { log.warn('\t', 'Warning, trading method has no name'); + } - if(!config.debug || !this.log) - this.log = function() {}; +// if(!config.debug || !this.log) +// DO NOT nuke the log for MACD logging, let line-by-line tweaks happen +// this.log = function() {}; this.setup = true; -} +}; // teach our base trading method events var Util = require('util'); @@ -75,10 +95,13 @@ Base.prototype.tick = function(candle) { // update all indicators var price = candle[this.priceValue]; _.each(this.indicators, function(i) { - if(i.input === 'price') + if(i.input === 'price') { i.update(price); - if(i.input === 'candle') + } + + if(i.input === 'candle') { i.update(candle); + } }); this.update(candle); @@ -90,29 +113,32 @@ Base.prototype.tick = function(candle) { // update previous price this.lastPrice = price; -} +}; Base.prototype.addIndicator = function(name, type, parameters) { - if(!_.contains(allowedIndicators, type)) + if(!_.contains(allowedIndicators, type)) { util.die('I do not know the indicator ' + type); + } - if(this.setup) + if(this.setup) { util.die('Can only add indicators in the init method!'); + } this.indicators[name] = new Indicators[type].factory(parameters); // some indicators need a price stream, others need full candles this.indicators[name].input = Indicators[type].input; -} +}; Base.prototype.advice = function(newPosition) { - if(!newPosition) + if(!newPosition) { return this.emit('soft advice'); + } this.emit('advice', { recommandation: newPosition, portfolio: 1 }); -} +}; -module.exports = Base; \ No newline at end of file +module.exports = Base; diff --git a/core/candleManager.js b/core/candleManager.js index 1687fb96a..3e35330af 100644 --- a/core/candleManager.js +++ b/core/candleManager.js @@ -68,7 +68,12 @@ var equals = util.equals; // even though we have leap seconds, every // day has the same amount of minutes -var MINUTES_IN_DAY = 1439; + +// these are NOT the same thing!!! +// -- kuzetsa, 2014 July 5th +var MINUTES_IN_DAY = 1440; +var START_OF_LAST_MINUTE_IN_DAY = 1439; + var Manager = function() { _.bindAll(this); @@ -94,15 +99,17 @@ var Manager = function() { // 'realtime updating' // make sure the historical folder exists - if(!fs.existsSync(this.historyPath)) + if(!fs.existsSync(this.historyPath)) { fs.mkdirSync(this.historyPath); - - if(!fs.existsSync(this.historyPath)) + } + + if(!fs.existsSync(this.historyPath)) { throw 'Gekko is unable to save historical data, do I have sufficient rights?'; + } this.on('processed', function() { log.debug('Processed trades, sleeping for', this.fetch.next.m.fromNow(true) + '...'); - }) + }); // .on('full history build', this.emitRealtime) }; @@ -114,31 +121,39 @@ Util.inherits(Manager, EventEmitter); Manager.prototype.requiredDays = function(timespan, from) { var days = []; - if(!from) - var start = this.current.day.clone(); - else - var start = from.clone(); +// variable declaration style cleanup +// to make sure we pass jslint / jshint + + var start = {}; + + if(!from) { + start = this.current.day.clone(); + } else { + start = from.clone(); + } + var to = start.clone().subtract('m', timespan).startOf('day'); while(start >= to) { days.push(this.mom(start.clone())); start.subtract('d', 1); - }; + } return days; -} +}; Manager.prototype.checkHistory = function() { log.debug('checking history'); - this.history = {} + this.history = {}; this.history.required = tradingAdvisor.enabled; - if(this.history.required) + if(this.history.required) { this.history.timespan = tradingAdvisor.candleSize * tradingAdvisor.historySize; - else + } else { // load a only the current day this.history.timespan = 0; + } this.history.days = this.requiredDays(this.history.timespan); // load them @@ -153,13 +168,14 @@ Manager.prototype.checkHistory = function() { this.processHistoryStats(); }, this) ); -} +}; // create a new daily database and load it Manager.prototype.createDatabase = function(mom) { - if(mom.dayString in this.days && this.days[mom.dayString].handle) + if(mom.dayString in this.days && this.days[mom.dayString].handle) { return log.debug('Skipping creation of already loaded database', mom.dayString); - + } + var filename = this.databaseName(mom); if(this.databaseExists(filename)) { @@ -189,12 +205,13 @@ Manager.prototype.createDatabase = function(mom) { day.handle.ensureIndex({fieldName: 's', unique: true}); this.days[day.string] = day; -} +}; // load a daily database and store it in this.days Manager.prototype.loadDatabase = function(mom) { - if(mom.dayString in this.days && this.days[mom.dayString].mounted) + if(mom.dayString in this.days && this.days[mom.dayString].mounted) { return log.debug('Skipping loading', mom.dayString, ', it already exists'); + } var filename = this.databaseName(mom); @@ -217,8 +234,9 @@ Manager.prototype.loadDatabase = function(mom) { this.days[day.string] = day; - if(!this.databaseExists(day)) + if(!this.databaseExists(day)) { return; + } day.mounted = true; day.exists = true; @@ -230,36 +248,41 @@ Manager.prototype.loadDatabase = function(mom) { } catch(e) { util.die('Database file ' + filename + ' is corrupt, delete or rename that file and try again.'); } -} +}; Manager.prototype.databaseExists = function(day) { - if(_.isString(day)) + if(_.isString(day)) { // we have a filename instead day = {filename: day}; + } - if(!day.filename) + if(!day.filename) { // we have a mom instead day.filename = this.databaseName(day); + } return fs.existsSync(day.filename); -} +}; // calculate stats of a daily database and store state // in this.days Manager.prototype.setDatabaseMeta = function(mom, cb) { var day = this.days[mom.dayString]; - if(!day.exists) + if(!day.exists) { return cb(); + } day.handle.find({s: {$gt: -1}}, _.bind(function(err, minutes) { - if(err) + if(err) { return cb(err); + } day.minutes = _.size(minutes); - if(!day.minutes) + if(!day.minutes) { return cb(); + } // this day has candles, store stats day.empty = false; @@ -272,7 +295,7 @@ Manager.prototype.setDatabaseMeta = function(mom, cb) { cb(); }, this)); -} +}; Manager.prototype.deleteDay = function(day, safe) { log.debug( @@ -280,17 +303,20 @@ Manager.prototype.deleteDay = function(day, safe) { day.string, 'as corrupt' ); - if(safe) + + if(safe) { fs.renameSync( day.filename, day.filename.replace('.db', '') + ' [incomplete ~ ' + (+moment()) + '].db' ); - else + } else { fs.unlinkSync(day.filename); + } - if(day.string in this.days) + if(day.string in this.days) { delete this.days[day.string]; -} + } +}; // loop through this.days and check every meta to // see what history we have available @@ -310,40 +336,55 @@ Manager.prototype.checkDaysMeta = function(err, results) { var isFirstDay = equals(this.current.day, day.time); - if(!day.exists) + if(!day.exists) { return false; - if(day.empty) + } + + if(day.empty) { return this.deleteDay(day); - if(!day.minutes) - return false; + } - if(!isFirstDay && day.endCandle.s !== MINUTES_IN_DAY) - // this day doesn't end at midnight + if(!day.minutes) { return false; + } + if(!isFirstDay && day.endCandle.s !== START_OF_LAST_MINUTE_IN_DAY) { + // the day REALLY DOES end at midnight, however... + // the last minute is 1 minute before midnight + // kuzetsa, 2014 July 5th + return false; + } + // we have content available.minutes += day.minutes; available.first = this.mom(day.start); - if(isFirstDay) + if(isFirstDay) { available.last = this.mom(day.end); + } // if current day it needs to go back to // midnight if we want to consider next day - if(isFirstDay && day.startCandle.s !== 0) + if(isFirstDay && day.startCandle.s !== 0) { return false; + } // if not current day, it needs to be full - if(!isFirstDay && !day.full) + if(!isFirstDay && !day.full) { return false; + } // this day is approved, up to next day return true; + + // this inline callback function was a mess + // but at least now passes jslint / jshint + // -- kuzetsa, 2014 June 29 }, this); this.history.available = available; -} +}; @@ -370,7 +411,11 @@ Manager.prototype.processHistoryStats = function() { // how many more minutes do we need? history.toFetch = history.timespan - history.available.minutes; - if(!history.toFetch < 1) { + // (!A < B) is confusing + // (A >= B) is easier to read + // and actually passes jshint / jslint + //-- kuzetsa, 2014 June 29 + if(history.toFetch >= 1) { // we have appear to have full history // though we need to verify on fetch // that we don't have a gap @@ -388,14 +433,15 @@ Manager.prototype.processHistoryStats = function() { completeAt: done }; - if(!history.available.minutes) + if(!history.available.minutes) { state.empty = true; + } this.history.missing = history.toFetch; state = _.extend(state, history.available); this.emit('history state', state); -} +}; Manager.prototype.setFetchMeta = function(data) { @@ -404,7 +450,7 @@ Manager.prototype.setFetchMeta = function(data) { end: this.mom(data.end), next: this.mom(utc().add('ms', data.nextIn)) }; -} +}; // we need to verify after the first fetch @@ -424,14 +470,16 @@ Manager.prototype.checkHistoryAge = function(data) { return; } - if(history.available.last.minute === MINUTES_IN_DAY) + if(history.available.last.minute === START_OF_LAST_MINUTE_IN_DAY) { this.increaseDay(); + } this.minumum = history.available.last.m.clone().add('m', 1); - if(this.minumum > this.fetch.start.m) + if(this.minumum > this.fetch.start.m) { // we're all good, process normally return; + } // there is a gap, mark current day as corrupted and process log.warn('The history we found is to old, we have to build a new one'); @@ -440,26 +488,32 @@ Manager.prototype.checkHistoryAge = function(data) { // reset available history history.available.gap = true; - history.available.minutes = 0 + history.available.minutes = 0; history.available.first = false; history.toFetch = history.timespan; -} +}; // Calculate when we have enough historical data // to provide normal advice Manager.prototype.calculateAdviceTime = function(next, args) { - if(this.state !== 'building history') + if(this.state !== 'building history') { // we are not building history, either // we don't need to or we're already done return next(args); + } var history = this.history; var toFetch = history.toFetch; - if(toFetch < 0) + // this notation seems confusing + // what does "less than zero" + // even mean in this context? + //-- kuzetsa, 2014 June 29 + if(toFetch < 0) { // we have full history return this.broadcastHistory(next, args); + } var doneAt = this.startTime.clone().add('m', toFetch + 1).startOf('minute').local(); // we have partly history @@ -474,19 +528,24 @@ Manager.prototype.calculateAdviceTime = function(next, args) { ); next(args); -} +}; // broadcast full history Manager.prototype.broadcastHistory = function(next, args) { var history = this.history; - if(history.available.last) + //variable declaration style cleanup + //to make sure we pass jslint / jshint + var last = {}; + var m = {}; + + if(history.available.last) { // first run - var last = this.mom(history.available.last.m); - else { + last = this.mom(history.available.last.m); + } else { // after filling full history while fetching live - var m = this.minuteToMoment(this.mostRecentCandle.s, this.current.day) - var last = this.mom(m); + m = this.minuteToMoment(this.mostRecentCandle.s, this.current.day); + last = this.mom(m); } var first = this.mom(last.m.clone().subtract('m', history.timespan)); @@ -503,23 +562,26 @@ Manager.prototype.broadcastHistory = function(next, args) { // get the candles for each required day var iterator = function(mom, next) { var from = 0; - var to = MINUTES_IN_DAY; + var to = START_OF_LAST_MINUTE_IN_DAY; - if(equals(mom.day, last.day)) + if(equals(mom.day, last.day)) { // on first (most recent) day to = last.minute; + } - if(equals(mom.day, first.day)) + if(equals(mom.day, first.day)) { // on last (oldest) day from = first.minute; + } this.getCandles(mom, from, to, next); - } + }; // emit all candles together var emit = function(err, batches) { - if(err || !batches) + if(err || !batches) { throw err; + } // transport batches = _.map(batches, function(batch, i) { @@ -539,14 +601,14 @@ Manager.prototype.broadcastHistory = function(next, args) { }); next(args); - } + }; async.map( days, _.bind(iterator, this), _.bind(emit, this) ); -} +}; // grab candles in a specific day // @@ -563,7 +625,7 @@ Manager.prototype.getCandles = function(mom, from, to, cb) { $gte: from } }, cb); -} +}; // grab a batch of trades and for each full minute // create a new candle @@ -571,8 +633,9 @@ Manager.prototype.processTrades = function(data) { this.setFetchMeta(data); - if(this.fetch.start.day > this.current.day) + if(this.fetch.start.day > this.current.day) { util.die('FATAL: Fetch data appears from the future, don\'t know how to process'); + } // if first run if(!this.minumum) { @@ -584,6 +647,7 @@ Manager.prototype.processTrades = function(data) { this.calculateAdviceTime(this.processTrades, data); return; } + var trades = this.filterTrades(data.all); if(!_.size(trades)) { @@ -591,7 +655,7 @@ Manager.prototype.processTrades = function(data) { return this.emit('processed'); } - log.debug('processing', _.size(trades), 'trade(s)'); + log.info('processing', _.size(trades), 'trade(s)'); log.debug( 'from', moment.unix(_.first(trades).date).utc().format(), @@ -601,24 +665,31 @@ Manager.prototype.processTrades = function(data) { var candles = this.calculateCandles(trades); + // variable declaration style cleanup + // to make sure we pass jslint / jshint + var ghostCandle = {}; + var batch = {}; + var batches = []; + if(this.fetchHasNewDay()) { // multple days - var batches = this.splitCandleDays(candles); + batches = this.splitCandleDays(candles); - if(_.size(batches) > 2) + if(_.size(batches) > 2) { util.error('More than 2 days of new data per fetch not supported'); + } // we are dealing with two days log.debug('fetch spans midnight'); // old day - batches[0] = this.addEmtpyCandles(_.first(batches), this.mostRecentCandle, MINUTES_IN_DAY); + batches[0] = this.addEmtpyCandles(_.first(batches), this.mostRecentCandle, START_OF_LAST_MINUTE_IN_DAY); // new day // we have to create a fake candle so we can fill gap // after midnight with empty candles reflecting last price - var ghostCandle = _.clone(_.last(batches[0])); + ghostCandle = _.clone(_.last(batches[0])); ghostCandle.s = -1; batches[1] = this.addEmtpyCandles(_.last(batches), ghostCandle); @@ -631,38 +702,45 @@ Manager.prototype.processTrades = function(data) { // we already know they are all from current day // but if the most recent candle is from yesterday ... - if(this.mostRecentCandle && this.mostRecentCandle.s === MINUTES_IN_DAY) { - var ghostCandle = _.clone(this.mostRecentCandle); + if(this.mostRecentCandle && this.mostRecentCandle.s === START_OF_LAST_MINUTE_IN_DAY) { + ghostCandle = _.clone(this.mostRecentCandle); ghostCandle.s = -1; - var batch = this.addEmtpyCandles(candles, ghostCandle); - } else - var batch = this.addEmtpyCandles(candles, this.mostRecentCandle); + batch = this.addEmtpyCandles(candles, ghostCandle); + } else { + batch = this.addEmtpyCandles(candles, this.mostRecentCandle); + } this.leftovers = batch.pop(); - var batches = [ batch ]; - }; + batches = [ batch ]; + } // set threshold for next fetch this.minumum = this.fetch.end.m.clone(); // if we only have leftovers we can skip insert clause - if(_.size(batches) === 1 && _.size(_.first(batches)) === 0) + if(_.size(batches) === 1 && _.size(_.first(batches)) === 0) { return this.finishInsert(); - + } // now we have candles in batches per day, insert and process result async.eachSeries(batches, this.insertCandles, this.finishInsert); -} +}; // filter out all trades that are do not belong to the last candle Manager.prototype.filterTrades = function(trades) { log.debug('minimum trade treshold:', this.minumum.utc().format('YYYY-MM-DD HH:mm:ss'), 'UTC'); - return trades = _.filter(trades, function(trade) { + trades = _.filter(trades, function(trade) { return this.minumum < moment.unix(trade.date).utc(); }, this); -} + + // it is unlear what (when / how) we + // return due to the async callback + //-- kuzetsa, 2014 June 29 + + return trades; +}; // turn a batch of trades into 1m candles. Is sorted as // long as the batch of trades are. @@ -722,7 +800,7 @@ Manager.prototype.calculateCandles = function(trades) { }); return candles; -} +}; // split a batch of candles into seperate arrays per day Manager.prototype.splitCandleDays = function(candles) { @@ -730,8 +808,9 @@ Manager.prototype.splitCandleDays = function(candles) { var last = 0; _.each(candles, function(c) { - if(c.s < last) + if(c.s < last) { batches.push([]); + } last = c.s; @@ -739,7 +818,7 @@ Manager.prototype.splitCandleDays = function(candles) { }, this); return batches; -} +}; @@ -756,13 +835,14 @@ Manager.prototype.splitCandleDays = function(candles) { // add empty candles. End is a minute # since midnight. // // for example: -// addEmtpyCandles(candles, 0, MINUTES_IN_DAY) +// addEmtpyCandles(candles, 0, START_OF_LAST_MINUTE_IN_DAY) // would return an array of candles from: // [midnight up to][batch without gaps][up to next midnight - 1] Manager.prototype.addEmtpyCandles = function(candles, start, end) { var length = _.size(candles); - if(!length) + if(!length) { return candles; + } if(start) { // put the start candle in front @@ -779,17 +859,24 @@ Manager.prototype.addEmtpyCandles = function(candles, start, end) { // if this was last and we don't // have to fill a gap after this batch // we're done - if(i === last && !end) + if(i === last && !end) { return; + } var min = c.s + 1; - if(i === last && end) - var max = end + 1; - else - var max = candles[i + 1].s; + //variable declaration style cleanup + //to make sure we pass jslint / jshint + var max = 0; - var empty, prevClose; + if(i === last && end) { + max = end + 1; + } else { + max = candles[i + 1].s; + } + + var empty = {}; + var prevClose = 0; if(min > max) { console.log('c', candles, 's', start, 'e', end); @@ -801,7 +888,7 @@ Manager.prototype.addEmtpyCandles = function(candles, start, end) { empty.s = min; empty.v = 0; - var prevClose = empty.c; + prevClose = empty.c; // set o, h, l, p to previous // close price empty.o = prevClose; @@ -816,21 +903,23 @@ Manager.prototype.addEmtpyCandles = function(candles, start, end) { // we added start to fill the gap from before // previous candles to this batch - if(start) + if(start) { candles.shift(); + } var all = candles.concat(emptyCandles); return this.sortCandles(all); -} +}; // store a batch of candles under the day specified by // `this.current`. If cb is specified runs this after // inserting all candles (inserts 1 per tick). Manager.prototype.insertCandles = function(candles, cb) { - if(!_.size(candles)) + if(!_.size(candles)) { return cb(); + } // because this function is async make // sure we are using the right day. @@ -845,7 +934,7 @@ Manager.prototype.insertCandles = function(candles, cb) { var iterator = function(c, next) { // async func wrapper to not pass error // and halt the async chain - var _next = function() { next() }; + var _next = function() { next(); }; this.defer(function() { log.debug( @@ -873,15 +962,16 @@ Manager.prototype.insertCandles = function(candles, cb) { next(); }); - } + }; var done = function() { - if(this.processMidnight) + if(this.processMidnight) { this.increaseDay(); + } this.processMidnight = false; cb(); - } + }; this.mostRecentCandle = _.last(candles); @@ -905,17 +995,18 @@ Manager.prototype.insertCandles = function(candles, cb) { ); }, this)); -} +}; // unmount a day from memory and mark the cache // ready for GC Manager.prototype.unloadDay = function(dayString) { - if(!this.days[dayString]) + if(!this.days[dayString]) { return; + } this.days[dayString].mounted = false; this.days[dayString].handle = false; -} +}; Manager.prototype.increaseDay = function() { log.debug('shifting past midnight'); @@ -924,31 +1015,33 @@ Manager.prototype.increaseDay = function() { this.unloadDay(this.current.dayString); this.setDay(this.current.day.clone().add('d', 1)); -} +}; Manager.prototype.finishInsert = function(err, results) { this.emit('processed'); -} +}; // // HELPERS Manager.prototype.setDay = function(m) { this.current = this.mom(m.clone()); -} +}; // how many days are in this trade batch? Manager.prototype.fetchHasNewDay = function() { return !equals(this.fetch.end.day, this.current.day); -} +}; Manager.prototype.sortCandles = function(candles) { return candles.sort(function(a, b) { return a.s - b.s; }); -} +}; Manager.prototype.minuteToMoment = function(minutes, day) { - if(!day) + if(!day) { day = this.current.day; + } + day = day.clone().startOf('day'); return day.clone().add('minutes', minutes); }; @@ -966,12 +1059,13 @@ Manager.prototype.mom = function(m) { minute: this.momentToMinute(m), day: m.clone().startOf('day'), dayString: m.format('YYYY-MM-DD') - } -} + }; +}; Manager.prototype.databaseName = function(mom) { - if(!('dayString' in mom)) + if(!('dayString' in mom)) { mom = this.mom(mom); + } return this.historyPath + [ config.watch.exchange, @@ -979,14 +1073,15 @@ Manager.prototype.databaseName = function(mom) { config.watch.asset, mom.dayString + '.db' ].join('-'); -} +}; // small wrapper around a fake candle // to make it easier to throw them around Manager.prototype.transportCandle = function(c, day) { // check whether we got a mom or a moment - if(!day.m) + if(!day.m) { day = this.mom(day.clone()); + } delete c._id; @@ -994,18 +1089,19 @@ Manager.prototype.transportCandle = function(c, day) { candle: c, time: day.m.clone().add('m', c.s), day: day - } -} + }; +}; + Manager.prototype.transportCandles = function(candles, day) { return _.map(candles, function(c) { return this.transportCandle(c, day); }, this); -} +}; // executes cb on next tick while // maintaining context (to `Manager`). Manager.prototype.defer = function(cb) { return _.defer(_.bind(cb, this)); -} +}; module.exports = Manager; diff --git a/core/candleStore.js b/core/candleStore.js index 352ecc358..1789c0672 100644 --- a/core/candleStore.js +++ b/core/candleStore.js @@ -15,7 +15,7 @@ var Day = function(day) { this.state = "uninitialized"; this.candles = []; this.filename = "history-" + day.toString() + ".csv"; -} +}; Day.prototype.addCandles = function(candles) { this.candles = this.candles.concat(candles); @@ -42,12 +42,12 @@ var Store = function() { this.unzip, this.readFile ); -} +}; Store.prototype.openDay = function(day, callback) { // Load only if the open day changed, or we never opened a day - if(this.day == null || day != this.day.day) { - prepareNewDay(day); + if(this.day === null || day !== this.day.day) { + this.prepareNewDay(day); this.loadDay(function(err, candles) { if(!err) { this.day.addCandles(candles); @@ -56,41 +56,41 @@ Store.prototype.openDay = function(day, callback) { callback(err, candles); }); } -} +}; Store.prototype.loadDay = function(day, callback) { this.read(day.filename, function(candles) { callback(null, candles); }); -} +}; Store.prototype.prepareNewDay = function(day) { - if(this.day.state != 'loading') { + if(this.day.state !== 'loading') { // Do we need to keep this.day.state = 'closing'; this.day = new Day(day); } -} +}; // Queue's candles to be added as soon as a day is loaded Store.prototype.addCandles = function(candles) { //NOTE: this.queue is array of arrays. this.queue.push(candles); this.flush(); -} +}; // If there is a day in open state, append all queued candles to it. Store.prototype.flush = function() { //TODO(yin): Help, this.day.state can get easily stuck locked. - if(this.queue.length > 0 && this.day != null && this.day.state = 'open') { + if(this.queue.length > 0 && this.day !== null && this.day.state === 'open') { this.day.addCandles(_.flatten(this.queue)); this.queue = []; this.day.state = 'saving'; this.write(this.day.filename, this.day.candles, function(err) { this.day.state = 'open'; - }) + }); } -} +}; Store.prototype.toCSV = function(file, candles, next) { var csv = _.map(candles, function(properties) { @@ -98,31 +98,31 @@ Store.prototype.toCSV = function(file, candles, next) { }).join('\n'); next(null, file, csv); -} +}; Store.prototype.deflate = function(file, csv, next) { zlib.deflate(csv, function(err, buffer) { next(err, file, buffer); }); -} +}; Store.prototype.writeFile = function(file, gzip, next) { fs.writeFile(this.directory + file, gzip, function(err) { next(err); }); -} +}; Store.prototype.readFile = function(file, next) { fs.readFile(this.directory + file, function(err, buffer) { next(err, buffer); }); -} +}; Store.prototype.unzip = function(buffer, next) { zlib.unzip(buffer, function(err, buffer) { next(err, buffer.toString()); }); -} +}; Store.prototype.toArray = function(csv, next) { var f = parseFloat; @@ -136,13 +136,13 @@ Store.prototype.toArray = function(csv, next) { l: f(l[3]), c: f(l[4]), p: f(l[5]) - } + }; }); next(obj); -} +}; //TODO(yin):Exported for tests -Store.Day = Day +Store.Day = new Day(); module.exports = Store; \ No newline at end of file diff --git a/core/portfolioManager.js b/core/portfolioManager.js index b34aac57f..edad38b67 100644 --- a/core/portfolioManager.js +++ b/core/portfolioManager.js @@ -28,6 +28,8 @@ var Manager = function(conf) { var Exchange = require('../exchanges/' + this.exchangeSlug); this.exchange = new Exchange(conf); + this.limiter = (new Date()).getTime(); + this.conf = conf; this.portfolio = {}; this.fee; @@ -118,11 +120,22 @@ Manager.prototype.getBalance = function(fund) { // is this a infinityOrderExchange (does it support order // requests bigger then the current balance?) Manager.prototype.trade = function(what) { - if(what !== 'BUY' && what !== 'SELL') - return; +// lizards ALWAYS happen instead of buy orders this.action = what; + if(what !== 'BUY') + return; + + +// is it time yet? +var rightnow = (new Date()).getTime(); + if(rightnow <= this.limiter) + return; + +// ok, so reset the limiter +this.limiter = (new Date()).getTime() + 9000; + var act = function() { var amount, price; @@ -131,8 +144,13 @@ Manager.prototype.trade = function(what) { // do we need to specify the amount we want to buy? if(this.infinityOrderExchange) amount = 10000; - else - amount = this.getBalance(this.currency) / this.ticker.ask; + else { + amount = 1 - this.fee; + amount = (100000000 * amount * this.getBalance(this.currency)) - 2; + amount = Math.floor(Math.floor(amount) / this.ticker.ask) - 1; + amount = Math.max(0, amount); + amount /= 100000000; + } // can we just create a MKT order? if(this.directExchange) @@ -142,6 +160,7 @@ Manager.prototype.trade = function(what) { this.buy(amount, price); + } else if(what === 'SELL') { // do we need to specify the amount we want to sell? @@ -155,8 +174,10 @@ Manager.prototype.trade = function(what) { price = false; else price = this.ticker.bid; - - this.sell(amount, price); + + // LIZARD!!! + // this.sell(amount, price); + } }; async.series([ @@ -179,7 +200,7 @@ Manager.prototype.getMinimum = function(price) { Manager.prototype.buy = function(amount, price) { // sometimes cex.io specifies a price w/ > 8 decimals price *= 100000000; - price = Math.floor(price); + price = Math.ceil(price); price /= 100000000; var currency = this.getFund(this.currency); @@ -189,10 +210,10 @@ Manager.prototype.buy = function(amount, price) { // if not suficient funds if(amount > availabe) { return log.info( - 'Wanted to buy but insufficient', + 'Insufficient', this.currency, - '(' + availabe + ')', - 'at', + '(' + availabe.toFixed(8) + ')', + 'to BUY at', this.exchange.name ); } @@ -200,18 +221,17 @@ Manager.prototype.buy = function(amount, price) { // if order to small if(amount < minimum) { return log.info( - 'Wanted to buy', - this.asset, - 'but the amount is too small', - '(' + amount + ')', + 'Ignore tiny', + '(' + amount.toFixed(8) + ' ' + this.asset + ')', + 'BUY order', 'at', this.exchange.name ); } log.info( - 'Attempting too BUY', - amount, + 'Placing BUY order for', + amount.toFixed(8), this.asset, 'at', this.exchange.name @@ -225,7 +245,7 @@ Manager.prototype.buy = function(amount, price) { Manager.prototype.sell = function(amount, price) { // sometimes cex.io specifies a price w/ > 8 decimals price *= 100000000; - price = Math.ceil(price); + price = Math.floor(price); price /= 100000000; var minimum = this.getMinimum(price); @@ -236,7 +256,7 @@ Manager.prototype.sell = function(amount, price) { return log.info( 'Wanted to buy but insufficient', this.asset, - '(' + availabe + ')', + '(' + availabe.toFixed(8) + ')', 'at', this.exchange.name ); @@ -248,7 +268,7 @@ Manager.prototype.sell = function(amount, price) { 'Wanted to buy', this.currency, 'but the amount is too small', - '(' + amount + ')', + '(' + amount.toFixed(8) + ')', 'at', this.exchange.name ); @@ -296,7 +316,7 @@ Manager.prototype.checkOrder = function() { Manager.prototype.logPortfolio = function() { log.info(this.exchange.name, 'portfolio:'); _.each(this.portfolio, function(fund) { - log.info('\t', fund.name + ':', fund.amount); + log.info('\t', fund.name + ':', fund.amount.toFixed(8)); }); } @@ -312,6 +332,12 @@ Manager.prototype.recheckPortfolio = function() { // the asset (GHS) as we are assuming the value // of the asset will go up. Manager.prototype.enforcePosition = function() { + +log.info('refreshed', this.exchange.name, 'portfolio:'); +_.each(this.portfolio, function(fund) { + log.info('\t', fund.name + ':', fund.amount.toFixed(8)); +}); + if(this.action !== 'BUY') return; diff --git a/core/util.js b/core/util.js index 0636786b9..cbbd303d2 100644 --- a/core/util.js +++ b/core/util.js @@ -16,6 +16,7 @@ var util = { var configFile = path.resolve(util.getArgument('config') || __dirname + '/../config.js'); _config = require(configFile); + _config.resolvedLocation = configFile; return _config; }, // overwrite the whole config diff --git a/docs/internals/exchanges.md b/docs/internals/exchanges.md index b3c62ec89..2a1275308 100644 --- a/docs/internals/exchanges.md +++ b/docs/internals/exchanges.md @@ -8,7 +8,7 @@ Gekko arranges all communication about when assets need to be bought or sold bet When you add a new exchange to Gekko you need to expose an object that has methods to query the exchange. This exchange file needs to reside in `gekko/exchanges` and the filename is the slug of the exchange name + `.js`. So for example the exchange for Mt. Gox is explained in `gekko/exchanges/mtgox.js`. -It is advised to use a npm module to query an exchange. This will seperate the abstract API calls from the Gekko specific stuff (In the case of Bitstamp there was no module yet, so I [created one](https://github.com/askmike/bitstamp)). +It is advised to use a npm module to query an exchange. This will seperate the abstract API calls from the Gekko specific stuff (In the case of Bitstamp, there was no module yet, so askmike [created one](https://github.com/askmike/bitstamp)). Besides this javascript file a new object needs to be added to `gekko/config.js` in the `config.traders` array. Here you can set the default currency and asset that will be available through the exchange. Keep enabled at false, this is up to the end user. diff --git a/exchanges.js b/exchanges.js index aceda7b69..7bdff7431 100644 --- a/exchanges.js +++ b/exchanges.js @@ -178,12 +178,36 @@ var exchanges = [ assets: ['GHS'], markets: [ { - pair: ['BTC', 'GHS'], minimalOrder: { amount: 0.000001, unit: 'currency' } + pair: ['BTC', 'GHS'], minimalOrder: { amount: 0.00000500, unit: 'currency' } } ], requires: ['key', 'secret', 'username'], providesHistory: false }, + { + name: 'Cryptsy', + slug: 'cryptsy', + direct: false, + infinityOrder: false, + currencies: ['BTC'], + assets: ['DOGE', 'DVC', 'PPC', 'LTC', 'DRK' ], + markets: [ + { + pair: ['BTC', 'DOGE'], market_id: 132, minimalOrder: { amount: 100, unit: 'asset' } + }, + { + pair: ['BTC', 'DVC'], market_id: 40, minimalOrder: { amount: 1, unit: 'asset' } + }, + { + pair: ['BTC', 'LTC'], market_id: 3, minimalOrder: { amount: 0.001, unit: 'asset' } + }, + { + pair: ['BTC', 'DRK'], market_id: 155, minimalOrder: { amount: 0.001, unit: 'asset' } + } + ], + requires: ['key', 'secret'], + providesHistory: false + }, { name: 'Kraken', slug: 'kraken', @@ -259,6 +283,23 @@ var exchanges = [ requires: ['key', 'secret'], monitorError: 'https://github.com/askmike/gekko/issues/210', providesHistory: false + }, + { + name: 'Bitfinex', + slug: 'bitfinex', + direct: false, + infinityOrder: false, + currencies: ['USD'], + assets: ['BTC'], + markets: [ + { + pair: ['USD', 'BTC'], minimalOrder: { amount: 0.01, unit: 'currency' } + } + ], + requires: ['key', 'secret'], + // TODO: should be possible to enable this for Bitfinex? + providesHistory: false + // fetchTimespan: 60 } ]; diff --git a/exchanges/cexio.js b/exchanges/cexio.js index e0ae470ab..0494597ba 100644 --- a/exchanges/cexio.js +++ b/exchanges/cexio.js @@ -9,7 +9,9 @@ var Trader = function(config) { this.user = config.username; this.key = config.key; this.secret = config.secret; - this.pair = 'ghs_' + config.currency.toLowerCase(); + this.currency = config.currency.toUpperCase(); + this.asset = config.asset.toUpperCase(); + this.pair = this.asset + '_' + this.currency; this.name = 'cex.io'; this.cexio = new CEXio( @@ -46,7 +48,7 @@ Trader.prototype.buy = function(amount, price, callback) { amount = Math.floor(amount); amount /= 100000000; - log.debug('BUY', amount, 'GHS @', price, 'BTC'); + log.debug('BUY', amount, this.asset, '@', price, this.currency); var set = function(err, data) { if(err) @@ -71,7 +73,7 @@ Trader.prototype.sell = function(amount, price, callback) { // test placing orders which will not be filled //price *= 10; price = price.toFixed(8); - log.debug('SELL', amount, 'GHS @', price, 'BTC'); + log.debug('SELL', amount, this.asset, '@', price, this.currency); var set = function(err, data) { if(err) @@ -149,7 +151,7 @@ Trader.prototype.getTicker = function(callback) { Trader.prototype.getFee = function(callback) { // cexio does currently don't take a fee on trades // TODO: isn't there an API call for this? - callback(false, 0.0); + callback(false, 0.002); } Trader.prototype.checkOrder = function(order, callback) { diff --git a/gekko.js b/gekko.js index 8fa5b5f80..2cf23ce53 100644 --- a/gekko.js +++ b/gekko.js @@ -34,10 +34,10 @@ if(util.getArgument('v')) { // make sure the current node version is recent enough if(!util.recentNode()) util.die([ - 'Your local version of nodejs is to old. ', + 'Your local version of nodejs is too old. ', 'You have ', process.version, - ' and you need atleast ', + ' and you need at least ', util.getRequiredNodeVersion() ].join('')); @@ -72,6 +72,9 @@ if( log.info('Gekko v' + util.getVersion(), 'started'); log.info('I\'m gonna make you rich, Bud Fox.', '\n\n'); +// Tell what config file we are using: +log.info('Using config file: ' + config.resolvedLocation); + var gekkoMode = 'realtime'; // currently we only support a single @@ -245,4 +248,4 @@ async.series( // everything is setup! emitters.market.start(); } -); \ No newline at end of file +); diff --git a/methods/MACD.js b/methods/MACD.js index 26168efba..8efeb3fab 100644 --- a/methods/MACD.js +++ b/methods/MACD.js @@ -4,6 +4,8 @@ (updated a couple of times since, check git history) + Modified by kuzetsa 2014 June 26 (CEXIO lizards variant) + */ // helpers @@ -28,9 +30,7 @@ method.init = function() { // report it. this.trend = { direction: 'none', - duration: 0, - persisted: false, - adviced: false + duration: 0 }; // how many candles do we need as a base @@ -57,11 +57,11 @@ method.log = function() { var signal = macd.signal.result; log.debug('calculated MACD properties for candle:'); - log.debug('\t', 'short:', macd.short.result.toFixed(digits)); - log.debug('\t', 'long:', macd.long.result.toFixed(digits)); - log.debug('\t', 'macd:', diff.toFixed(digits)); - log.debug('\t', 'signal:', signal.toFixed(digits)); - log.debug('\t', 'macdiff:', macd.result.toFixed(digits)); + log.info('\t', 'short:', macd.short.result.toFixed(digits)); + log.info('\t', 'long:', macd.long.result.toFixed(digits)); + log.info('\t', 'macd:', diff.toFixed(digits)); + log.info('\t', 'signal:', signal.toFixed(digits)); + log.info('\t', 'macdiff:', macd.result.toFixed(digits)); } method.check = function() { @@ -80,68 +80,25 @@ method.check = function() { // reset the state for the new trend this.trend = { duration: 0, - persisted: false, direction: 'up', - adviced: false }; this.trend.duration++; - log.debug('In uptrend since', this.trend.duration, 'candle(s)'); - - if(this.trend.duration >= settings.thresholds.persistence) - this.trend.persisted = true; + log.info('In uptrend since', this.trend.duration, 'candle(s)'); - if(this.trend.persisted && !this.trend.adviced) { - this.trend.adviced = true; this.advice('long'); - } else - this.advice(); - } else if(macddiff < settings.thresholds.down) { + } else { - // new trend detected - if(this.trend.direction !== 'down') - // reset the state for the new trend this.trend = { duration: 0, - persisted: false, - direction: 'down', - adviced: false + direction: 'lizards', }; - this.trend.duration++; - - log.debug('In downtrend since', this.trend.duration, 'candle(s)'); - - if(this.trend.duration >= settings.thresholds.persistence) - this.trend.persisted = true; - - if(this.trend.persisted && !this.trend.adviced) { - this.trend.adviced = true; - this.advice('short'); - } else - this.advice(); - - } else { - - log.debug('In no trend'); - - // we're not in an up nor in a downtrend - // but for now we ignore sideways trends - // - // read more @link: - // - // https://github.com/askmike/gekko/issues/171 - // this.trend = { - // direction: 'none', - // duration: 0, - // persisted: false, - // adviced: false - // }; + this.advice('lizards'); - this.advice(); } } diff --git a/methods/PPO.js b/methods/PPO.js index d70710bc8..7914d6415 100644 --- a/methods/PPO.js +++ b/methods/PPO.js @@ -4,6 +4,8 @@ (updated a couple of times since, check git history) + Modified by kuzetsa 2014 June 26 (CEXIO lizards variant) + */ // helpers @@ -23,9 +25,7 @@ method.init = function() { this.trend = { direction: 'none', - duration: 0, - persisted: false, - adviced: false + duration: 0 }; this.requiredHistory = config.tradingAdvisor.historySize; @@ -52,14 +52,14 @@ method.log = function() { var ppoSignal = ppo.PPOsignal.result; log.debug('calculated MACD properties for candle:'); - log.debug('\t', 'short:', short.toFixed(digits)); - log.debug('\t', 'long:', long.toFixed(digits)); - log.debug('\t', 'macd:', macd.toFixed(digits)); - log.debug('\t', 'macdsignal:', macdSignal.toFixed(digits)); - log.debug('\t', 'machist:', (macd - macdSignal).toFixed(digits)); - log.debug('\t', 'ppo:', result.toFixed(digits)); - log.debug('\t', 'pposignal:', ppoSignal.toFixed(digits)); - log.debug('\t', 'ppohist:', (result - ppoSignal).toFixed(digits)); + log.info('\t', 'short:', short.toFixed(digits)); + log.info('\t', 'long:', long.toFixed(digits)); + log.info('\t', 'macd:', macd.toFixed(digits)); + log.info('\t', 'macdsignal:', macdSignal.toFixed(digits)); + log.info('\t', 'machist:', (macd - macdSignal).toFixed(digits)); + log.info('\t', 'ppo:', result.toFixed(digits)); + log.info('\t', 'pposignal:', ppoSignal.toFixed(digits)); + log.info('\t', 'ppohist:', (result - ppoSignal).toFixed(digits)); } method.check = function() { @@ -83,68 +83,25 @@ method.check = function() { if(this.trend.direction !== 'up') this.trend = { duration: 0, - persisted: false, - direction: 'up', - adviced: false + direction: 'up' }; this.trend.duration++; - log.debug('In uptrend since', this.trend.duration, 'candle(s)'); - - if(this.trend.duration >= settings.thresholds.persistence) - this.trend.persisted = true; + log.info('In uptrend since', this.trend.duration, 'candle(s)'); - if(this.trend.persisted && !this.trend.adviced) { - this.trend.adviced = true; this.advice('long'); - } else - this.advice(); - - } else if(ppoHist < settings.thresholds.down) { - // new trend detected - if(this.trend.direction !== 'down') + } else { + this.trend = { duration: 0, - persisted: false, - direction: 'down', - adviced: false + direction: 'lizards' }; - this.trend.duration++; - - log.debug('In downtrend since', this.trend.duration, 'candle(s)'); - - if(this.trend.duration >= settings.thresholds.persistence) - this.trend.persisted = true; - - if(this.trend.persisted && !this.trend.adviced) { - this.trend.adviced = true; - this.advice('short'); - } else - this.advice(); - - - } else { - - log.debug('In no trend'); - - // we're not in an up nor in a downtrend - // but for now we ignore sideways trends - // - // read more @link: - // - // https://github.com/askmike/gekko/issues/171 - // this.trend = { - // direction: 'none', - // duration: 0, - // persisted: false, - // adviced: false - // }; + this.advice('lizards'); - this.advice(); } } diff --git a/methods/indicators/EMA.js b/methods/indicators/EMA.js index 1d3d50a0b..c6c9dbd97 100644 --- a/methods/indicators/EMA.js +++ b/methods/indicators/EMA.js @@ -1,4 +1,4 @@ -// @link http://en.wikipedia.org/wiki/Exponential_moving_average#Exponential_moving_average +// @link http://en.wikipedia.org/wiki/Exponential_moving_average#Exponential_moving_average var Indicator = function(weight) { this.weight = weight; diff --git a/package.json b/package.json index ba0b56f60..3000495ec 100644 --- a/package.json +++ b/package.json @@ -12,12 +12,14 @@ }, "author": "Mike van Rossum ", "dependencies": { + "cryptsy-api": "0.1.x", "mtgox-apiv2": "1.1.x", "lodash": "2.x", "moment": "2.4.x", "btc-e": "0.0.x", "cexio": "0.0.x", "bitstamp": "0.1.x", + "bitfinex": "git+https://github.com/naddison36/bitfinex.git", "async": "0.2.x", "nedb": "0.9.4", "line-reader": "0.2.x", @@ -30,6 +32,6 @@ "engines": { "node": ">=0.10.x" }, "repository": { "type": "git", - "url": "https://github.com/askmike/gekko.git" + "url": "https://github.com/kuzetsa/gekko.git" } } diff --git a/plugins.js b/plugins.js index b6c39957b..979cab1e6 100644 --- a/plugins.js +++ b/plugins.js @@ -78,6 +78,30 @@ var actors = [ version: '0.1.1' }] }, + { + name: 'Mandrill Mailer', + description: 'Mandrill Mail module lets sends you email yourself everytime Gekko has new advice.', + slug: 'mandrillMailer', + async: true, + silent: false, + modes: ['realtime'], + dependencies: [{ + module: 'mandrill-api', + version: '1.0.40' + }] + }, + { + name: 'SMS Plivo', + description: 'SMS module to text yourself everytime Gekko has new advice. Uses Plivo.', + slug: 'smsPlivo', + async: true, + silent: false, + modes: ['realtime'], + dependencies: [{ + module: 'plivo', + version: '0.1.0' + }] + }, { name: 'Trader', description: 'Trader will follow the advice and create real orders.', diff --git a/plugins/adviceLogger.js b/plugins/adviceLogger.js index 29bb2f9f4..52cdfbd85 100644 --- a/plugins/adviceLogger.js +++ b/plugins/adviceLogger.js @@ -14,12 +14,12 @@ Actor.prototype.processTrade = function(trade) { }; Actor.prototype.processAdvice = function(advice) { - console.log() - log.info('We have new trading advice!'); - log.info('\t Position to take:', advice.recommandation); - log.info('\t Market price:', this.price); - log.info('\t Based on market time:', this.marketTime.format('YYYY-MM-DD HH:mm:ss')); - console.log() + // console.log() + log.debug('We have new trading advice!'); + log.debug('\t Position to take:', advice.recommandation); + log.info('\t Market price:', Number(this.price).toFixed(8)); + log.info('\t At timestamp:', this.marketTime.format('YYYY-MM-DD HH:mm:ss')); + // console.log() }; -module.exports = Actor; \ No newline at end of file +module.exports = Actor; diff --git a/plugins/trader.js b/plugins/trader.js index 9d9f35e84..174fc49c2 100644 --- a/plugins/trader.js +++ b/plugins/trader.js @@ -13,19 +13,14 @@ var Trader = function(next) { Trader.prototype.processAdvice = function(advice) { if(advice.recommandation == 'long') { this.manager.trade('BUY'); - log.info( + log.debug( 'Trader', 'Received advice to go long', 'Buying ', config.trader.asset ); - } else if(advice.recommandation == 'short') { - this.manager.trade('SELL'); - log.info( - 'Trader', - 'Received advice to go short', - 'Selling ', config.trader.asset - ); - } + } else + this.manager.trade('LIZARD'); +// lizards are not going cause a sell order. EVER!!! } module.exports = Trader; diff --git a/plugins/tradingAdvisor.js b/plugins/tradingAdvisor.js index 78e4f00cc..8cb50b06f 100644 --- a/plugins/tradingAdvisor.js +++ b/plugins/tradingAdvisor.js @@ -5,7 +5,11 @@ var _ = require('lodash'); var config = util.getConfig(); var methods = [ + 'ZERO', 'MACD', + 'x2MACD', + 'nikiehihsa', + 'x3nikiehihsa', 'DEMA', 'PPO', 'RSI', @@ -17,8 +21,9 @@ var Actor = function() { var methodName = config.tradingAdvisor.method; - if(!_.contains(methods, methodName)) + if(!_.contains(methods, methodName)) { util.die('Gekko doesn\'t know the method ' + methodName); + } var Consultant = require('../core/baseTradingMethod'); @@ -30,11 +35,11 @@ var Actor = function() { Consultant.prototype[name] = fn; }); - this.method = new Consultant; -} + this.method = new Consultant(); +}; Actor.prototype.processCandle = function(candle) { this.method.tick(candle); -} +}; module.exports = Actor;