elective-stereophonic
elective-stereophonic
Any way to get all transactions since a particular block index? singapore
Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Stable Nxt Client: Nxt 1.12.2

Author Topic: Any way to get all transactions since a particular block index?  (Read 1832 times)

monsterer

  • Full Member
  • ***
  • Karma: +18/-2
  • Offline Offline
  • Posts: 175
    • View Profile

I was surprised to find there didn't seem to be an API call for getting all transactions on an account since a particular block? You can do it by timestamp, but that seems a little bit handwavey.

I'm not sure getting ALL transactions and manually parsing them is going to be great for performance.
Logged
https://metaexchange.info/
NXT<->BTC instant exchange - low spread, no registration

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: Any way to get all transactions since a particular block index?
« Reply #1 on: June 24, 2015, 02:22:47 pm »

I was surprised to find there didn't seem to be an API call for getting all transactions on an account since a particular block? You can do it by timestamp, but that seems a little bit handwavey.

I'm not sure getting ALL transactions and manually parsing them is going to be great for performance.
getAccountTransactions has a timestamp field
just pass in timestamp of the block you want to start from

alternatively just iterate firstIndex=0, lastIndex=0, then 1, then 2, ... until you hit the block you want
I have code that inverts this firstindex to a canonical index where the firsttx is the lowest txindex and it increments from there

James
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

monsterer

  • Full Member
  • ***
  • Karma: +18/-2
  • Offline Offline
  • Posts: 175
    • View Profile
Re: Any way to get all transactions since a particular block index?
« Reply #2 on: June 24, 2015, 02:32:30 pm »

getAccountTransactions has a timestamp field
just pass in timestamp of the block you want to start from

alternatively just iterate firstIndex=0, lastIndex=0, then 1, then 2, ... until you hit the block you want
I have code that inverts this firstindex to a canonical index where the firsttx is the lowest txindex and it increments from there

James

What if two blocks share the same timestamp?

Method two results in N separate API calls, doesn't it?
Logged
https://metaexchange.info/
NXT<->BTC instant exchange - low spread, no registration

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: Any way to get all transactions since a particular block index?
« Reply #3 on: June 24, 2015, 02:50:28 pm »

getAccountTransactions has a timestamp field
just pass in timestamp of the block you want to start from

alternatively just iterate firstIndex=0, lastIndex=0, then 1, then 2, ... until you hit the block you want
I have code that inverts this firstindex to a canonical index where the firsttx is the lowest txindex and it increments from there

James

What if two blocks share the same timestamp?

Method two results in N separate API calls, doesn't it?
I have yet to see two blocks at the same timestamp.
these calls are done locally, so you can do thousands per second.
if speed is an issue you can cache them
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

monsterer

  • Full Member
  • ***
  • Karma: +18/-2
  • Offline Offline
  • Posts: 175
    • View Profile
Re: Any way to get all transactions since a particular block index?
« Reply #4 on: June 24, 2015, 02:53:30 pm »

I have yet to see two blocks at the same timestamp.
these calls are done locally, so you can do thousands per second.
if speed is an issue you can cache them

That doesn't fill me with confidence. I'd like to know *for sure* that it was impossible before I design systems around it...

In addition, it's not speed I'm concerned about, but failure handling. It's much more of a PITA to deal with failure with multiple calls.
Logged
https://metaexchange.info/
NXT<->BTC instant exchange - low spread, no registration

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: Any way to get all transactions since a particular block index?
« Reply #5 on: June 24, 2015, 03:26:38 pm »

I have yet to see two blocks at the same timestamp.
these calls are done locally, so you can do thousands per second.
if speed is an issue you can cache them

That doesn't fill me with confidence. I'd like to know *for sure* that it was impossible before I design systems around it...

In addition, it's not speed I'm concerned about, but failure handling. It's much more of a PITA to deal with failure with multiple calls.
just make one call then
leave off firstIndex and lastIndex
put in the right timestamp

not sure what is so PITA about this, the data is there and in the right order for what you need
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: Any way to get all transactions since a particular block index?
« Reply #6 on: June 27, 2015, 04:09:19 pm »

Using timestamp to retrieve only transactions not yet seen is a problem indeed, when the timestamp is recent enough and there is a chance that a fork switch may have occurred since the last call. This is a common issue when using almost all APIs, if you assume certain state of the blockchain before calling the API. I am thinking of the following solution, accept an optional "knownBlock" parameter, and if present, check that this block is indeed present in the blockchain, return an error if not, and include the current lastBlock id in the response. Then if you want to make sure there was no pop-off or switch to another fork since the last time you polled for transactions, add the lastBlock id from the previous request as knownBlock parameter. Or the block id corresponding to the last processed timestamp, to be sure that nothing before that has changed.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: Any way to get all transactions since a particular block index?
« Reply #7 on: June 28, 2015, 08:13:18 am »

I added requireBlock and requireLastBlock optional parameters to all APIs where it makes sense. If either of those is supplied, the API will return an error immediately if the required block is not found in the blockchain, or the current last block is not the required last block; it will also execute in a synchronized block, to make sure the blockchain state does not change in the meantime, and will include the current last block id in the response. Using those, you can reliably use timestamps to fetch only not yet seen transactions without having to detect possible fork switches or pop-offs as a special case (will get an error if such have occurred).

A note about timestamps, the block timestamp used in queries on the transaction table (such as getBlockchainTransactions, getAccountTransactions) is the timestamp of the block in which the transaction is included. The block timestamp of derived objects such as Trades is the timestamp of the block in which the transaction is executed. For phased transactions, those two are different!
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322
 

elective-stereophonic
elective-stereophonic
assembly
assembly