elective-stereophonic
elective-stereophonic
number of queries per second possbile to NRS singapore
Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Stable Nxt Client: Nxt 1.11.15 | Latest Experimental Nxt Client: Nxt 1.12.0e

Author Topic: number of queries per second possbile to NRS  (Read 1760 times)

LocoMB

  • Hero Member
  • *****
  • Karma: +101/-37
  • Offline Offline
  • Posts: 751
    • View Profile
number of queries per second possbile to NRS
« on: November 14, 2015, 02:59:51 pm »


I want to make an app that does large numbers of  queries on the NRS backend.

Since I have to do it non-blocking anyway, I have to dump each query into a thread, so I can do that very fast, and not incrementally after each returned reply form the NRS.

There is a spam filter built into the NRS that limits the queires in order to avoid potentially malicious overloading of the NRS.

By queueing up a large number of queries and simply dumping them on the NRS in a loop, this spam filter is triggered- I remember that from last year when I did something similar.

Can anbody please give me some more information on how many queries I can send to the NRS per second or how long the time interval between two queries should be in order to not have them lost?

Maybe @Jean-Luc or @Riker?

I vaguely remember 50ms in between two queries- add to that a bit of safety margin, and an app should not make more than 15 queries per second, is this still correct?

Thanks!
Logged
TOX
90E54E5B5213290EE616D425CADC473038CFABFA53C913271AA8559D1937DC4AF3A354A9E4E5

verymuchso

  • Hero Member
  • *****
  • Karma: +118/-2
  • Offline Offline
  • Posts: 549
    • View Profile
    • HEAT Ledger
Re: number of queries per second possbile to NRS
« Reply #1 on: November 14, 2015, 04:06:49 pm »

There is a request throtler on the peer server (@see https://github.com/fimkrypto/nxt-plus/blob/master/src/java/nxt/peer/Peers.java#L381) but not on the API server.
There is no need for an API throttler since you can simply turn that off.

What you want is a pure Java connection to NRS, where you can 'spoof' HTTP requests to NRS from code.

For NXT+ we provide API access to the standard API through a websocket connector that translates API calls.
@see https://github.com/fimkrypto/nxt-plus/blob/master/src/java/nxt/http/rpc/CallAPIFunction.java, something similar should be used if you want to hit the API server while bypassing the network so to say.
You could easily do 1000 requests per second that way (depending on what calls you make of course - unlikely you can do 1000 blockchain rescans  ;D).

Hope this helps.
Logged
HEAT: DEX | SDK | HOME

LocoMB

  • Hero Member
  • *****
  • Karma: +101/-37
  • Offline Offline
  • Posts: 751
    • View Profile
Re: number of queries per second possbile to NRS
« Reply #2 on: November 15, 2015, 05:57:40 am »

Thanks, Yes!

I think I will have to start with keeping it under the throttle limit, because I can't do java...

Code: [Select]
if (Nxt.getBooleanProperty("nxt.enablePeerServerDoSFilter")) {
                    FilterHolder dosFilterHolder = peerHandler.addFilterWithMapping(DoSFilter.class, "/*", FilterMapping.DEFAULT);
                    dosFilterHolder.setInitParameter("maxRequestsPerSec", Nxt.getStringProperty("nxt.peerServerDoSFilter.maxRequestsPerSec"));
                    dosFilterHolder.setInitParameter("delayMs", Nxt.getStringProperty("nxt.peerServerDoSFilter.delayMs"));
                    dosFilterHolder.setInitParameter("maxRequestMs", Nxt.getStringProperty("nxt.peerServerDoSFilter.maxRequestMs"));
                    dosFilterHolder.setInitParameter("trackSessions", "false");
                    dosFilterHolder.setAsyncSupported(true);

hmm.. now I need to find where those values are acutally set... - I should do a grep on the repo for 'maxRequestsPerSec' or 'delayMs' ...
« Last Edit: November 15, 2015, 06:03:23 am by LocoMB »
Logged
TOX
90E54E5B5213290EE616D425CADC473038CFABFA53C913271AA8559D1937DC4AF3A354A9E4E5

verymuchso

  • Hero Member
  • *****
  • Karma: +118/-2
  • Offline Offline
  • Posts: 549
    • View Profile
    • HEAT Ledger
Re: number of queries per second possbile to NRS
« Reply #3 on: November 15, 2015, 11:18:15 am »

Nxt.getStringProperty("nxt.peerServerDoSFilter.maxRequestsPerSec"));

Means its value is set in nxt.properties file and it expects a string.
Put these in nxt.properties file..

nxt.peerServerDoSFilter.maxRequestsPerSec=NUMBER
nxt.peerServerDoSFilter.delayMs=NUMBER

But it will not help make the API server accept more calls. API server and Peer server are two separate things.


Logged
HEAT: DEX | SDK | HOME

LocoMB

  • Hero Member
  • *****
  • Karma: +101/-37
  • Offline Offline
  • Posts: 751
    • View Profile
Re: number of queries per second possbile to NRS
« Reply #4 on: November 15, 2015, 12:06:03 pm »

are the defaults are in nxt-properties?

I don't need to crank it up, I only have to know the value that it is set to to have a starting point..
Logged
TOX
90E54E5B5213290EE616D425CADC473038CFABFA53C913271AA8559D1937DC4AF3A354A9E4E5

Riker

  • Core Dev
  • Hero Member
  • *****
  • Karma: +439/-42
  • Offline Offline
  • Posts: 1796
    • View Profile
Re: number of queries per second possbile to NRS
« Reply #5 on: November 15, 2015, 02:14:07 pm »

are the defaults are in nxt-properties?

I don't need to crank it up, I only have to know the value that it is set to to have a starting point..

The default values for DOSFilter are specified in conf/nxt-default.properties as follows:
Code: [Select]
# Settings for the Jetty Denial Of Service Filter, used for the peer networking
# server only.
nxt.peerServerDoSFilter.maxRequestsPerSec=30
nxt.peerServerDoSFilter.delayMs=1000
nxt.peerServerDoSFilter.maxRequestMs=300000
The DOS filter is enabled by default:
Code: [Select]
# Enable the Jetty Denial of Service Filter for the peer networking server.
nxt.enablePeerServerDoSFilter=true

The main parameter is maxRequestsPerSec which is self explanatory.
The other parameters are explained here http://www.eclipse.org/jetty/documentation/current/dos-filter.html
Logged
NXT Core Dev
Account: NXT-HBFW-X8TE-WXPW-DZFAG
Public Key: D8311651 Key fingerprint: 0560 443B 035C EE08 0EC0  D2DD 275E 94A7 D831 1651

LocoMB

  • Hero Member
  • *****
  • Karma: +101/-37
  • Offline Offline
  • Posts: 751
    • View Profile
Re: number of queries per second possbile to NRS
« Reply #6 on: November 15, 2015, 05:26:05 pm »


Thanks Guys!

Very much appreciated!  :)
Logged
TOX
90E54E5B5213290EE616D425CADC473038CFABFA53C913271AA8559D1937DC4AF3A354A9E4E5
 

elective-stereophonic
elective-stereophonic
assembly
assembly