v4.1.0: release to master (#1069)

DeviaVir · web-flow · commit f235ba1e3966 · 2018-01-07T02:32:12.000+01:00
* bring the recent commits to master down into unstable (#1048) * update products.json file to newest trading pairs at binance (#1028) * Binance changed API 24 hours to 1 hour (#1027) * Update README * Update README again * add generation data to gitignore (#1012) * binance changed their API strategy from 24 hour windows to 1 hour. * rename periodLength to period_length (#1035) * Change markup_pct to new markdown and markup, set commonArgs so it is included in all strategies, fix return. (#1047) The only other occurrence is in the readme file noting the legacy aspect. * Update darwin.js * Added support for dema-strategy to darwin.js (#1046) * fix bad escaping on output to console (#1045) * Update periodLength to underscore syntax for darwin * Missing | lead to bitwise operation being performed rather than an OR logical comparison. (#1053) * Console Output adjustment (#1055) * resolution to .csv & .json output (#1058) * resolution to .csv & .json output trogdor123 got me looking at the right section where the params were being input in json. made some minor tweaks to the code and now both are outputting correctly. * Update darwin.js ...changes - formatting. * darwin.js - crossover strategy (#1060) noticed that generational output was static from gen to gen for smalen1 and smalen2. sure enough, those are coded into the strategy itself, but were left out of the darwin.js strategy config for mutation. * Docker (#1061) * docker fix * Resolve docker build problems * noop strategy: readme/links/list (#1062) * Travis. Docker. (#1063) * Updated minimum required node version (#1066) It appears Zenbot wont work on anything before 8.3.0. Updating the minimum required version, for clarity. * Merge DemaStrats (#1067) fixes #1064 * v4.1.0: prepare release (#1068) * v4.1.0: prepare release * update package-lock.json
diff --git a/.travis.yml b/.travis.yml
@@ -0,0 +1,8 @@
+language: node_js
+node_js:
+  - "8"
+  - "9"
+before_install:
+  - rm -rf package-lock.json
+  - npm install https://github.com/floatdrop/pinkie-promise.git
+  - npm install -g node-gyp
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -0,0 +1,44 @@
+## Changelog
+
+- [v4.1.0](https://github.com/carlos8f/zenbot/releases/tag/v4.1.0) (Latest)
+    - more indicators
+    - more strategies
+    - more exchanges
+    - too many bug fixes to list here
+    - web UI
+    - Travis
+    - Docker automated builds
+    - Automated tests
+- [v4.0.5](https://github.com/carlos8f/zenbot/releases/tag/v4.0.5)
+    - handle insufficient funds errors from gdax
+    - new trend_ema defaults adjusted for latest btc movements: 20m period, neutral_rate=0
+    - include more data in sim output
+    - remove rarely useful trend_ema options
+    - avoid abort in trader on failed getTrades()
+- v4.0.4
+    - debugging for polo odd results
+    - sim: simplify and correct makerFee assessment
+    - fix conf path in API credentials errors
+    - fix order total under 0.0001 error on polo
+    - Docker: extend README slightly (thanks [@DeviaVir](https://github.com/deviavir) and [@egorbenko](https://github.com/egorbenko))
+    - docker-compose: do not expose mongodb by default! (thanks [@DeviaVir](https://github.com/deviavir))
+- v4.0.3
+    - fix for docker mongo host error
+    - link for new Discord chat!
+    - fix polo crash on getOrder weird result
+    - fix oversold_rsi trigger while in preroll
+    - fix polo "not enough..." errors
+    - fancy colors for price report
+    - display product id in report
+    - fix poloniex backfill batches too big, mongo timeouts
+    - fix cursorTo() crash on some node installs
+    - memDump for debugging order failures
+    - fix column spacing on progress report
+- v4.0.2
+    - minor overhaul to trend_ema strat - added whipsaw filtering via std. deviation (`--neutral_rate=auto`)
+    - trim preroll of sim result graph
+- v4.0.1
+    - Added .dockerignore (thanks [@sulphur](https://github.com/sulphur))
+    - fix crashing on mongo timeout during backfill
+    - fix gaps in poloniex backfill
+    - default backfill days 90 -> 14
diff --git a/Dockerfile b/Dockerfile
@@ -4,7 +4,11 @@ RUN mkdir -p /app
 WORKDIR /app
 
 COPY package.json /app/
-RUN npm install -g node-gyp && npm install --unsafe-perm
+COPY webpack.config.js /app/
+COPY webpack-src /app/webpack-src
+COPY templates /app/templates
+RUN npm install -g node-gyp
+RUN npm install --unsafe-perm
 
 COPY . /app
 RUN ln -s /app/zenbot.sh /usr/local/bin/zenbot
diff --git a/README.md b/README.md
diff --git a/commands/trade.js b/commands/trade.js
@@ -387,9 +387,11 @@ module.exports = function container (get, set, clear) {
             get('db.trades').select(opts, function (err, trades) {
               if (err) throw err
               if (!trades.length) {
-                console.log('------------------------------------------ INITIALIZE  OUTPUT ------------------------------------------')
+                var head = '------------------------------------------ INITIALIZE  OUTPUT ------------------------------------------';
+                console.log(head)
                 get('lib.output').initializeOutput(s)
-                console.log('---------------------------- STARTING ' + so.mode.toUpperCase() + ' TRADING ----------------------------')
+                var minuses = Math.floor((head.length - so.mode.length - 19) / 2)
+                console.log('-'.repeat(minuses) + ' STARTING ' + so.mode.toUpperCase() + ' TRADING ' + '-'.repeat(minuses + (minuses % 2 == 0 ? 0 : 1)))
                 if (so.mode === 'paper') {
                   console.log('!!! Paper mode enabled. No real trades are performed until you remove --paper from the startup command.')
                 }
diff --git a/docker-compose-windows.yml b/docker-compose-windows.yml
@@ -1,5 +1,5 @@
 server:
-  build: .
+  image: deviavir/zenbot:unstable
   volumes:
     - ./conf.js:/app/conf.js
     - ./extensions:/app/extensions
@@ -12,7 +12,7 @@ mongodb:
   image: mongo:latest
   volumes_from:
     - mongodb-data
-  command: mongod --smallfiles
+  command: mongod --smallfiles --bind_ip=0.0.0.0 --logpath=/dev/null
 
 mongodb-data:
   image: mongo:latest
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -3,7 +3,7 @@ version: '3.1'
 services:
 
   server:
-    build: .
+    image: deviavir/zenbot:unstable
     volumes:
       - ./conf.js:/app/conf.js
       - ./extensions:/app/extensions
@@ -22,7 +22,7 @@ services:
     image: mongo:latest
     volumes:
       - ./data/db:/data/db
-    command: mongod --smallfiles --logpath=/dev/null
+    command: mongod --smallfiles --bind_ip=0.0.0.0 --logpath=/dev/null
     ports:
       - 27017:27017
 
diff --git a/extensions/exchanges/bittrex/exchange.js b/extensions/exchanges/bittrex/exchange.js
@@ -38,7 +38,7 @@ module.exports = function container(get, set, clear) {
       var timeout = 2500
     }
 
-    console.error(('\Bittrex API error - unable to call ' + method + ' (' + error.message + '), retrying in ' + timeout / 1000 + 's').red)
+    console.error(('\nBittrex API error - unable to call ' + method + ' (' + error.message + '), retrying in ' + timeout / 1000 + 's').red)
     setTimeout(function () {
       exchange[method].apply(exchange, args)
     }, timeout)
diff --git a/extensions/exchanges/poloniex/exchange.js b/extensions/exchanges/poloniex/exchange.js
@@ -73,7 +73,7 @@ module.exports = function container (get, set, clear) {
           return retry('getTrades', func_args)
         }
         if (!body.map) {
-          console.error('\getTrades odd result:')
+          console.error('\ngetTrades odd result:')
           console.error(body)
           return retry('getTrades', func_args)
         }
@@ -100,7 +100,7 @@ module.exports = function container (get, set, clear) {
           return retry('getBalance', args)
         }
         if (body.error) {
-          console.error('\ggetBalance error:')
+          console.error('\ngetBalance error:')
           console.error(body)
           return retry('getBalance', args)
         }
@@ -126,7 +126,7 @@ module.exports = function container (get, set, clear) {
           return retry('getQuote', args)
         }
         if (body.error) {
-          console.error('\ggetQuote error:')
+          console.error('\ngetQuote error:')
           console.error(body)
           return retry('getQuote', args)
         }
diff --git a/extensions/strategies/dema-lessnoise/_codemap.js b/extensions/strategies/dema-lessnoise/_codemap.js
diff --git a/extensions/strategies/dema-lessnoise/strategy.js b/extensions/strategies/dema-lessnoise/strategy.js
diff --git a/extensions/strategies/dema/strategy.js b/extensions/strategies/dema/strategy.js
@@ -15,6 +15,7 @@ module.exports = function container (get, set, clear) {
       this.option('down_trend_threshold', 'threshold to trigger a sold signal', Number, 0)
       this.option('overbought_rsi_periods', 'number of periods for overbought RSI', Number, 9)
       this.option('overbought_rsi', 'sold when RSI exceeds this value', Number, 80)
+      this.option('noise_level_pct', 'do not trade when short ema is with this % of last short ema', Number, 0)
     },
 
     calculate: function (s) {
@@ -47,7 +48,9 @@ module.exports = function container (get, set, clear) {
       }
 
       if (typeof s.period.dema_histogram === 'number' && typeof s.lookback[0].dema_histogram === 'number') {
-        if ((s.period.dema_histogram - s.options.up_trend_threshold) > 0 && (s.lookback[0].dema_histogram - s.options.up_trend_threshold) <= 0) {
+        if (s.options.noise_level_pct != 0 && (s.period.ema_short / s.lookback[0].ema_short * 100 < s.options.noise_level_pct)) {
+          s.signal = 'null';
+        } else if ((s.period.dema_histogram - s.options.up_trend_threshold) > 0 && (s.lookback[0].dema_histogram - s.options.up_trend_threshold) <= 0) {
           s.signal = 'buy';
         } else if ((s.period.dema_histogram + s.options.down_trend_threshold) < 0 && (s.lookback[0].dema_histogram + s.options.down_trend_threshold) >= 0) {
           s.signal = 'sell';
diff --git a/extensions/strategies/noop/_codemap.js b/extensions/strategies/noop/_codemap.js
@@ -1,5 +1,6 @@
 module.exports = {
   _ns: 'zenbot',
 
-  'strategies.noop': require('./strategy')
-}
+  'strategies.noop': require('./strategy'),
+  'strategies.list[]': '#strategies.noop'
+}
diff --git a/extensions/strategies/trendline/strategy.js b/extensions/strategies/trendline/strategy.js
@@ -72,8 +72,8 @@ module.exports = function container (get, set, clear) {
          s.signal = 'buy'
          }
       else if (
-         s.growth === false |
-         s.growth2 === false |
+         s.growth === false ||
+         s.growth2 === false ||
          s.accel === false
          )
          {
diff --git a/lib/engine.js b/lib/engine.js
@@ -706,7 +706,7 @@ module.exports = function container (get, set, clear) {
         process.stdout.write(z(8, pct(diff), ' ')[diff >= 0 ? 'green' : 'red'])
       }
       else {
-        process.stdout.write(z(8, '', ' '))
+        process.stdout.write(z(9, '', ' '))
       }
       var volume_display = s.period.volume > 99999 ? abbreviate(s.period.volume, 2) : n(s.period.volume).format('0')
       volume_display = z(8, volume_display, ' ')
@@ -789,8 +789,8 @@ module.exports = function container (get, set, clear) {
           z(19, 'DATE', ' ').grey,
           z(17, 'PRICE', ' ').grey,
           z(9, 'DIFF', ' ').grey,
-          z(15, 'VOL', ' ').grey,
-          z(12, 'RSI', ' ').grey,
+          z(10, 'VOL', ' ').grey,
+          z(8, 'RSI', ' ').grey,
           z(32, 'ACTIONS', ' ').grey,
           z(25, 'BAL', ' ').grey,
           z(22, 'PROFIT', ' ').grey
diff --git a/package-lock.json b/package-lock.json
diff --git a/package.json b/package.json
diff --git a/scripts/genetic_backtester/darwin.js b/scripts/genetic_backtester/darwin.js

Original file line number	Diff line number	Diff line change
`@@ -38,7 +38,7 @@ module.exports = function container(get, set, clear) {`
`38`	`38`	`var timeout = 2500`
`39`	`39`	`}`
`40`	`40`
`41`		`- console.error(('\Bittrex API error - unable to call ' + method + ' (' + error.message + '), retrying in ' + timeout / 1000 + 's').red)`
	`41`	`+ console.error(('\nBittrex API error - unable to call ' + method + ' (' + error.message + '), retrying in ' + timeout / 1000 + 's').red)`
`42`	`42`	`setTimeout(function () {`
`43`	`43`	`exchange[method].apply(exchange, args)`
`44`	`44`	`}, timeout)`
Original file line number	Diff line number	Diff line change
`@@ -73,7 +73,7 @@ module.exports = function container (get, set, clear) {`
`73`	`73`	`return retry('getTrades', func_args)`
`74`	`74`	`}`
`75`	`75`	`if (!body.map) {`
`76`		`- console.error('\getTrades odd result:')`
	`76`	`+ console.error('\ngetTrades odd result:')`
`77`	`77`	`console.error(body)`
`78`	`78`	`return retry('getTrades', func_args)`
`79`	`79`	`}`
`@@ -100,7 +100,7 @@ module.exports = function container (get, set, clear) {`
`100`	`100`	`return retry('getBalance', args)`
`101`	`101`	`}`
`102`	`102`	`if (body.error) {`
`103`		`- console.error('\ggetBalance error:')`
	`103`	`+ console.error('\ngetBalance error:')`
`104`	`104`	`console.error(body)`
`105`	`105`	`return retry('getBalance', args)`
`106`	`106`	`}`
`@@ -126,7 +126,7 @@ module.exports = function container (get, set, clear) {`
`126`	`126`	`return retry('getQuote', args)`
`127`	`127`	`}`
`128`	`128`	`if (body.error) {`
`129`		`- console.error('\ggetQuote error:')`
	`129`	`+ console.error('\ngetQuote error:')`
`130`	`130`	`console.error(body)`
`131`	`131`	`return retry('getQuote', args)`
`132`	`132`	`}`