Nxt Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Nxt Client 1.11.9 - NEW RELEASE: Ardor 2.0.3e TestNet IS LAUNCHED!

Pages: [1]

Author Topic: TA-JSON-RPC  (Read 1732 times)

nexern

  • Board Moderator
  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 496
    • View Profile
  • Karma: +83/-11
TA-JSON-RPC
March 01, 2015, 03:39:38 pm

first draft
the finhive ta-lib is based and the result on an intense research on all available ta resources found so far and unified into
a fast and lean C lib (current size ~250kb). it provides more than 400 indicators, candlesticks, formation and
calculation functions so far and is still growing. primitives are also included to create custom/compound indicators.

to map this powerfull lib into a data agnostic json-rpc* while providing a simple 'new module' plugin functionality, the draft
below shows the first approach to archives this. since it is a draft, it will change and evolve over time, so just take it as a
possible data flow concept. the goal is to get even very complex resultsets just with a single api call. sectionally arrays are
used to prevent endless api repetitions and pipelining allows to create powerfull custom indicators and signaling.

as an  example a full call could look like this:
-----------------------------------------------------------------------------------------------------

{
  "input": { "mode": 1, "ex" : "ex_bitfx", "instrument": "usd_btc", "frame": "d1", "start": 1420070400, "end": 0, "data": [] },
 
  "indicators": {
    "ind_trima": { "series": "c", "term": 14 },
    "ind_bollb": { "series": "b", "term": 20, "multi": 2 },
    "ind_rsi"  : { "series": "o", "term": 7 }
    ...
  },
 
  "formations": {
    "cdl": [ "cdl_advb", "cdl_djll" ],
    "frm": [ "frm_kru", "frm_gap" ]
    ...
  },
 
  "calculations": {
    "clc": [ "clc_ath" ]
    ...
  },
 
  "classifiers": {
    "svm": [ "frm_dcbs" ],
    "asm": [ "frm_dcbs" ]
    ...
  },
 
  "pipes": {
    "ind_rsi": { "to": "ind_rsqr", "term": 14, "as": "myInd1" },
    "myInd1" : { "to": "ind_ema" , "term": 21, "as": "myInd2" }
    ...
  },
 
  "triggers": [ "ind_bollb", "ind_trima", "ind_rsi", "myInd2" ],
 
  "trades": { ... ruleset, not defined. boolean sufficient or better dsl? ... },
 
  "out": {
    "input"       : [ "dohlcv" ],
    "indicators"  : [ "ind_trima","ind_bollb" ],
    "formations"  : [ "all"],
    "classifiers" : [ "svm", "asm" ],
    "pipes"       : [ "myInd2" ],
    "triggers"    : [ "all" ],
    "trades"      : []
  }

}

-----------------------------------------------------------------------------------------------------

input:
mode can be 1 or 2. 1 = use interal data provided by finhive. 2 = use your own data. needs to fill up
the data array in bar (d) ohlc (v) format. this example uses internal data. bitfinix usd_btc, 1day
starting at 1420070400 unix timestamp.

indicators:
array of indicators, indicators specs (ids, terms, input, output etc.) will be released the next days.
series indicates on what the indicator will be applied. in this case the trima (Triangular Moving Average)
needs an array, eg. close = c, while the bollingers needs the full bars = b.

formations:
detect candlestick and bar formations (boolean/fuzzy approach) on input and deliver
same input size ( 0,0,0,0,1,0,0,0 ) array hits. hits can be resolved by input timestamp.

calculations:
add all time high on input into output stream or other pure statistical vals.

classifiers:
run predefined classifiers on 'unsharp pattern' and deliver approx. hit percentage.
this section needs more testing because it would make very much sense to allow
classifiers to run also on a sliced data windows, which could be used to identify
unfinished formation approx. the problem here is the cpu load it would generate. this
needs more testing later but i added it as a concept to make the flow more complete.
classifiers will be added with the final version.

pipes:
pipe any series into another one. very powerfull to create own indicators, signaling and more.
in this example (not very usefull btw but ok) the rsi is piped into a linear regression and then
smoothed by an 21 ema pipe.

triggers:
triggers are used for signaling. triggers will check for standard threshold values, stored within
the ta encyclopedia. for the rsi, the flow would be this:
the rsi ranges from 0 to 100, so the rsi is interpreted as an overbought/oversold indicator when
the value is over 70 or below 30. triggers output would report an buy/sell signal if those thresholds
are reached or none for unknown/custom indicators.

trades:
not defined now but this section will provide backtesting functionality. the problem i am facing here
is the combinatorical explosion and how to overcome this in an elegant manner. the best solution would
be a complete new dsl but this is pretty much work. atm it seems a simple boolean approach could be
sufficient for the start.

out:
compose the result output.

-------------------------------------------
ind_bollb   - Bollinger Bands
ind_trima   - Triangular Moving Average
ind_rsi      - Relative Strength Index
ind_pfe     - Polarized Fractal Efficiency
ind_rsqr    - R-Squared
cdl_advb    - Candlestick >> Advanced Block formation
cdl_djll     - Candlestick >> Doji, long leg
frm_kru     - Key reversal, uptrend
frm_gap      - Gap formation
frm_dcbs   - Dead cat bounce standard formation
clc_ath      - All time high
-------------------------------------------

*reason for the api change is to add an additonal revenue stream (at least the possibility ) to finhive.
besides bot-devs, financial data/portal owners or software client devs could use this api using their own data on it.

ps: feedback very welcome, this approach isn't optimal i know but sometimes you can't see the forest for the trees.
« Last Edit: March 01, 2015, 06:51:11 pm by nexern »

jl777

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 6176
    • View Profile
  • Karma: +718/-123
Re: TA-JSON-RPC
March 01, 2015, 03:43:01 pm

I made InstantDEX API calls:

static char *allsignals[] = { (char *)allsignals_func, "allsignals", "V", 0 };
static char *getsignal[] = { (char *)getsignal_func, "getsignal", "V", "signal", "start", "end", "resolution", "baseid", "relid", "base", "rel", 0 };
the allsignals API currently just returns hardcoded: {"signals":[{"name":"SMA","scale":"price"},{"name":"slope","scale":"auto"}]}
just an array of ("name" "scale") with "price" meaning same as the price and "auto" meaning it needs to be scaled by the display probably in separate area from price

to get a signal you would construct a getsignal API with "signal" being then name returned from allsignals, start and end are unixtimes in seconds, resolution is defaulting to 60 but it can be set to anything, then the usual baseid/relid which would be for a specific orderbook or base/rel which would be for a higher level blend of all base/rel orderbooks

the returned JSON will have a "values" array of 5 item arrays [timestamp, OHLC], Open High Low CLose (edited)

So I think we can map the TA calls to the signals

James
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

nexern

  • Board Moderator
  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 496
    • View Profile
  • Karma: +83/-11
Re: TA-JSON-RPC
March 01, 2015, 04:06:28 pm

I made InstantDEX API calls:

static char *allsignals[] = { (char *)allsignals_func, "allsignals", "V", 0 };
static char *getsignal[] = { (char *)getsignal_func, "getsignal", "V", "signal", "start", "end", "resolution", "baseid", "relid", "base", "rel", 0 };
the allsignals API currently just returns hardcoded: {"signals":[{"name":"SMA","scale":"price"},{"name":"slope","scale":"auto"}]}
just an array of ("name" "scale") with "price" meaning same as the price and "auto" meaning it needs to be scaled by the display probably in separate area from price

to get a signal you would construct a getsignal API with "signal" being then name returned from allsignals, start and end are unixtimes in seconds, resolution is defaulting to 60 but it can be set to anything, then the usual baseid/relid which would be for a specific orderbook or base/rel which would be for a higher level blend of all base/rel orderbooks

the returned JSON will have a "values" array of 5 item arrays [timestamp, OHLC], Open High Low CLose (edited)

So I think we can map the TA calls to the signals

James

harhar, pretty fast james. i am on the draft and writing some explanation now. should be ready in 30 min.
i am not satisfied with my current version and would like to get some feedback. perhaps we can discuss it.

jl777

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 6176
    • View Profile
  • Karma: +718/-123
Re: TA-JSON-RPC
March 01, 2015, 06:51:55 pm

I think the interfaces are pretty close. A few tweaks to make it able to just drop-in.

I have all the contract pairs specified as base/rel (baseid/relid)
where base -> rel at the specified price. with base NXT and rel BTC -> NXT/BTC

this allows the user to flip the display to their preferred orientation just by reversing the base and rel. It also allows for creating virtual prices for single currencies. [I have proven formulas to make predictive virtual prices for a currency in the abstract.]

My convention is to use assetid pairs for specific orderbooks (with native cryptos getting assetid == string as bits), but if it is "base":"NXT","rel":"BTC" then this would indicate that it is the weighted average value of each. The biggest problem with making good predictors is to eliminate noise and this averaging is a key factor.

It is good you already have standard timeframes, so to map your API to mine would be a matter to predefine specific "signal" to pre-selected defaults. This can be an ever expanding list and also to allow the user to define any specific one to make a new signal and have this added to the allsignals return.

due to the interface to the GUI being one signal at a time, it looks like it would just be a simplified "out" field.

OK, so what we need is createsignal API at my level, which is simply passed in the template for what to send to your library. Then during initialization, we can set the allsignals API to the list of premade signals in a config file and the user can make any custom indicator using the same createsignal API.

The GUI will use the allsignals API to make a list of signals to overlay.

Once we get the signals interfaced, it is just a matter to add the ability to issue actual orders based on the signals as inputs, with of course an arbitrary amount of NN, SVM, etc. to fine tune its risk/reward and expected ROI profile

If it is a C library, I can just link it into SuperNET

James
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

nexern

  • Board Moderator
  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 496
    • View Profile
  • Karma: +83/-11
Re: TA-JSON-RPC
March 01, 2015, 07:49:41 pm

I think the interfaces are pretty close. A few tweaks to make it able to just drop-in.

I have all the contract pairs specified as base/rel (baseid/relid)
where base -> rel at the specified price. with base NXT and rel BTC -> NXT/BTC

this allows the user to flip the display to their preferred orientation just by reversing the base and rel. It also allows for creating virtual prices for single currencies. [I have proven formulas to make predictive virtual prices for a currency in the abstract.]

My convention is to use assetid pairs for specific orderbooks (with native cryptos getting assetid == string as bits), but if it is "base":"NXT","rel":"BTC" then this would indicate that it is the weighted average value of each. The biggest problem with making good predictors is to eliminate noise and this averaging is a key factor.

It is good you already have standard timeframes, so to map your API to mine would be a matter to predefine specific "signal" to pre-selected defaults. This can be an ever expanding list and also to allow the user to define any specific one to make a new signal and have this added to the allsignals return.

due to the interface to the GUI being one signal at a time, it looks like it would just be a simplified "out" field.

OK, so what we need is createsignal API at my level, which is simply passed in the template for what to send to your library. Then during initialization, we can set the allsignals API to the list of premade signals in a config file and the user can make any custom indicator using the same createsignal API.

The GUI will use the allsignals API to make a list of signals to overlay.

Once we get the signals interfaced, it is just a matter to add the ability to issue actual orders based on the signals as inputs, with of course an arbitrary amount of NN, SVM, etc. to fine tune its risk/reward and expected ROI profile

If it is a C library, I can just link it into SuperNET

James

to accelerate let's move over to slack chat. i have some questions regarding the imput/synthetic-pairs you mentioned.

jl777

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 6176
    • View Profile
  • Karma: +718/-123
Re: TA-JSON-RPC
March 02, 2015, 01:22:45 am

Power outage and internet problems...
for some reason I cant log in to slack and I cant even ssh...

Probably have to wait for a while to get this all fixed, so I will just write more code

James

There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

annarosy

  • Newbie
  • *
  • Offline Offline
  • Posts: 4
    • View Profile
    • scr888
  • Karma: +0/-0
Re: TA-JSON-RPC
July 01, 2016, 03:49:37 am

 interfaces are pretty close.Then during initialization, we can set the allsignals API to the list of premade signals in a config file and the user can make any custom indicator using the same createsignal API.
Pages: [1]