elective-stereophonic
elective-stereophonic
NXTservices_div test release for dividend calculations at any block  
Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Nxt Client: Nxt 1.11.15

Pages: [1] 2 3 ... 7  All

Author Topic: NXTservices_div test release for dividend calculations at any block  (Read 21323 times)

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile

http://209.126.71.170/NXTservices_div1.zip is for Mac
http://209.126.71.170/NXTservices_div0_win.zip is for windows
THIS USES MAINNET but it does not need an acct or acct password

usage is:
./NXTservices_div0 DIV <assetid> <blocknum>

If you leave off both assetid and blocknum, it will do the calculation for all assets
lots of stuff will appear on the screen and it will create assetstats.txt with JSON for the NXT account balances for the corresponding asset

I am hoping we can get this validated so the asset owners out there can use it to get automated dividend processing. From the JSON data it should be pretty easy to loop through and sendMoney for the actual dividend process. I leave that task to somebody else. The hard part was getting the individual balances at any arbitrary blocknumber

Now, I only tested matching the balances with the latest block because I could use the getAccount API to verify it matches my calculations. One of the reasons this took so long is that the only way I can think of to verify dividend allocations at an arbitrary block is to manually verify it.

I suggest starting with assets that dont have many transactions

James
« Last Edit: May 19, 2014, 10:45:38 pm by jl777 »
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile

command was ./NXTservices_div1 DIV 144000 69320371311895680144800

logfile https://dl.dropboxusercontent.com/u/53047281/NXTservices1.log

didn't crash
no lib_curl messages
no asset...txt file
terminated after 15 mins
reverse the blocknum and assetid
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

chanc3r

  • Hero Member
  • *****
  • Karma: +124/-50
  • Offline Offline
  • Posts: 1019
  • NXTInspect
    • View Profile

Yeah sorry tired cut/paste error
So apart from obvious block range invalid asset id checking :)

If you enter a block after asset or account creation it seems to be ok
If you enter a block before the asset or account was created it seems to go wrong.

Sits there with this.
Code: [Select]
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
19) LTC t15249325 6952492416469182066 txid.14825870881683083128
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
27999: ave  1.868646 | elapsed 1.182 millis | NXT calls.(http://127.0.0.1:7876/nxt?requestType=getBlock&block=7618999001792634345)
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
Logged
NXT: 29996814460165 (NXT-JTA7-B2QR-8BFC-2V222)
@imrimr @NXTinspect

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile

Yeah sorry tired cut/paste error
So apart from obvious block range invalid asset id checking :)

If you enter a block after asset or account creation it seems to be ok
If you enter a block before the asset or account was created it seems to go wrong.

Sits there with this.
Code: [Select]
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
19) LTC t15249325 6952492416469182066 txid.14825870881683083128
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
27999: ave  1.868646 | elapsed 1.182 millis | NXT calls.(http://127.0.0.1:7876/nxt?requestType=getBlock&block=7618999001792634345)
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
send 600 bytes to socket.17
cmd.(requestType=getAccount&account=7928896704997062037) -> {"errorCode":5,"errorDescription":"Unknown account"}
I just want to make sure it is calculating asset ownership correctly before I wrapped an API around it
can you test the first version to see if the curl errors go away if the parameters are in the right order?

thanks
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile

I got all the trades sorted by NXT acct and can now see which accounts made how much for each asset. Some guys are doing pretty good trading on AE! Still need to properly calculate the average cost of current inventory and marking to the market to get a realtime portfolio profitability report. I think I will just have the user input how much they spent/received for each asset they care about. There is no reliable way to correlate NXT payment with asset transfer.

I was thinking about making a top AE traders list, maybe even a little contest.

but until I get thorough test results on the dividend calcs, I think I will hold off on a release

James
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

GQM

  • Sr. Member
  • ****
  • Karma: +27/-12
  • Offline Offline
  • Posts: 487
    • View Profile

I got all the trades sorted by NXT acct and can now see which accounts made how much for each asset. Some guys are doing pretty good trading on AE! Still need to properly calculate the average cost of current inventory and marking to the market to get a realtime portfolio profitability report. I think I will just have the user input how much they spent/received for each asset they care about. There is no reliable way to correlate NXT payment with asset transfer.

I was thinking about making a top AE traders list, maybe even a little contest.

but until I get thorough test results on the dividend calcs, I think I will hold off on a release

James

I have managed to get all the account numbers of current shareholders. This is great and will allow me to get dividends paid one way or another. Automated would be best but Rome wasn't built in a day.

Thank you for getting this done for the community :)
Logged

chanc3r

  • Hero Member
  • *****
  • Karma: +124/-50
  • Offline Offline
  • Posts: 1019
  • NXTInspect
    • View Profile

Done some more testing, now i've woken up...
Seems to work with valid values.
Assume you will fix for assets it can't find and block number higher than current.

Question - because of risk of  node on a forks shouldnt' we advise people to restart their NRS and use a minimum difference between the current block height and the dividend block height e.g. 1 day or 1440 or risk a list with errors?

I did some more specific testing with  silver and transferred some assets in and out of an account to make sure...

In the block before the transfer it does not appear in the list if the account has NEVER held it before.
If the account has held the asset you still emit the account number but with a balance of 0.00000 - which is perhaps a bit confusing.

Question: why list people who have ZERO for dividends? finding accounts who have ever held the stock is useful though

so account 10728451002573970815
asset silver 7819056276221630295

block 141252 - pre-transfer balance is 0.00000 (but it has held it before so it is still in the results)
block 141253 - block in which the transfer TX is forged balance in results is 0.1 which is correct :)
block 141254 - block afterwards (for completeness)  balance in results is 0.1 which is correct :)

had a couple of examples where NXTservices stopped - i think this is because somehow it has upset NRS which also seems to have locked - but I have no idea why the high volume of API calls would cause this, after a few runs NRS does seem to become a bit 'lazy', something to watch for as more NXTservices is rolled out.
« Last Edit: May 20, 2014, 10:08:31 am by chanc3r »
Logged
NXT: 29996814460165 (NXT-JTA7-B2QR-8BFC-2V222)
@imrimr @NXTinspect

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile

Done some more testing, now i've woken up...
Seems to work with valid values.
Assume you will fix for assets it can't find and block number higher than current.

Question - because of risk of  node on a forks shouldnt' we advise people to restart their NRS and use a minimum difference between the current block height and the dividend block height e.g. 1 day or 1440 or risk a list with errors?

I did some more specific testing with  silver and transferred some assets in and out of an account to make sure...

In the block before the transfer it does not appear in the list if the account has NEVER held it before.
If the account has held the asset you still emit the account number but with a balance of 0.00000 - which is perhaps a bit confusing.

Question: why list people who have ZERO for dividends? finding accounts who have ever held the stock is useful though

so account 10728451002573970815
asset silver 7819056276221630295

block 141252 - pre-transfer balance is 0.00000 (but it has held it before so it is still in the results)
block 141253 - block in which the transfer TX is forged balance in results is 0.1 which is correct :)
block 141254 - block afterwards (for completeness)  balance in results is 0.1 which is correct :)

had a couple of examples where NXTservices stopped - i think this is because somehow it has upset NRS which also seems to have locked - but I have no idea why the high volume of API calls would cause this, after a few runs NRS does seem to become a bit 'lazy', something to watch for as more NXTservices is rolled out.
You should be able to calculate the set of asset holders and the exact QNT they had as of any given block.
The further back in time, the more reliable it is, so at least 10 confirmations
To be super safe, 1440 blocks seems like a good rule, maybe that should be the minimum
Each asset would have its own profile, so if we are talking serious value being sent out then maybe even a couple of days might make sense. Assuming the asset holders are ok with that.
for fluffy things 10 blocks would be fine

I keep the zero balance accts cuz I was planning on calculating everyone's trading performance, cost basis of existing assets, etc., but nobody seems to really care about it so I will just leave it as it is

Did you verify I had actually killed NRS? If so, maybe its not a libcurl problem. I am having to add small sleeps between each curl call just so it doesnt keep getting errors and doing giant sleeps

I think something about the DB inside NRS cant handle high volume of requests.It has not survived a single loading of recent blockchain without generating an error.

James

P.S. All the crypto daemons seem to have this problem. Is my code just to fast for the NXT core to keep up?
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

chanc3r

  • Hero Member
  • *****
  • Karma: +124/-50
  • Offline Offline
  • Posts: 1019
  • NXTInspect
    • View Profile


Did you verify I had actually killed NRS? If so, maybe its not a libcurl problem. I am having to add small sleeps between each curl call just so it doesnt keep getting errors and doing giant sleeps


Agree with everything else you posted.

I didn't do any exhaustive testing of the stress NXTservices put on NRS, was doing this on the train into work :) I killed NRS process myself because at one point NXTservices was just hanging at start.

I suspect we need to do some controlled test of this to prove if there are any API performance issues @high volume to pass back to the NRS core team as this is probably the first time something is driving NRS API hard.

Its less of an issue for DIV calculations where you can run and restart but a bigger issue perhaps for MGW and other longer running services.
Logged
NXT: 29996814460165 (NXT-JTA7-B2QR-8BFC-2V222)
@imrimr @NXTinspect

sepehr

  • Hero Member
  • *****
  • Karma: +44/-6
  • Offline Offline
  • Posts: 810
  • CryptoCurrency Evangelist
    • View Profile

Oh I see this thread too late!
Just try it and let you know the results.

Thanks a lot for this script.
Logged

sepehr

  • Hero Member
  • *****
  • Karma: +44/-6
  • Offline Offline
  • Posts: 810
  • CryptoCurrency Evangelist
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #10 on: May 24, 2014, 11:22:11 am »

Gave me this error:

"The program can't start because LIBEAY32.dll is missing from your computer. Try reinstalling the program to fix this problem"

Please guide
Logged

valarmg

  • Hero Member
  • *****
  • Karma: +178/-57
  • Offline Offline
  • Posts: 1766
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #11 on: May 24, 2014, 02:23:38 pm »

Gave me this error:

"The program can't start because LIBEAY32.dll is missing from your computer. Try reinstalling the program to fix this problem"

Please guide
You need SSL on windows. Go here and download either the 32bit or 64bit version. You might need to download some of the C++ Redistributables to get it to work.

Then you may have problems with some other .dll files. msvcr110.dll or msvcr120.dll

You'll need to get them from here: http://www.dll-files.com/dllindex/dll-files.shtml?msvcr110
Logged
NXT-CSED-4PK5-AR4V-6UB5V

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #12 on: May 24, 2014, 10:37:09 pm »

Any Mac testers available tonight?
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

allwelder

  • Hero Member
  • *****
  • Karma: +196/-13
  • Offline Offline
  • Posts: 1867
  • NxtChina.org
    • View Profile
    • NxtChina.org
Re: NXTservices_div test release for dividend calculations at any block
« Reply #13 on: May 26, 2014, 12:07:06 am »

will this feature be added to client?
Logged
NxtChina |Weibo |Twitter Donation welcomed:NXT-APL9-66GU-K8LY-B3JJJ

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #14 on: May 26, 2014, 12:08:40 am »

will this feature be added to client?
It is in the API and I have asked longzai to add support in the enhanced GUI
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

allwelder

  • Hero Member
  • *****
  • Karma: +196/-13
  • Offline Offline
  • Posts: 1867
  • NxtChina.org
    • View Profile
    • NxtChina.org
Re: NXTservices_div test release for dividend calculations at any block
« Reply #15 on: May 26, 2014, 06:35:38 am »

will this feature be added to client?
It is in the API and I have asked longzai to add support in the enhanced GUI
good,great feature can be visited directly on client is important.
Logged
NxtChina |Weibo |Twitter Donation welcomed:NXT-APL9-66GU-K8LY-B3JJJ

sepehr

  • Hero Member
  • *****
  • Karma: +44/-6
  • Offline Offline
  • Posts: 810
  • CryptoCurrency Evangelist
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #16 on: May 27, 2014, 02:30:50 am »

Gave me this error:

"The program can't start because LIBEAY32.dll is missing from your computer. Try reinstalling the program to fix this problem"

Please guide
You need SSL on windows. Go here and download either the 32bit or 64bit version. You might need to download some of the C++ Redistributables to get it to work.

Then you may have problems with some other .dll files. msvcr110.dll or msvcr120.dll

You'll need to get them from here: http://www.dll-files.com/dllindex/dll-files.shtml?msvcr110

thanks a lot for your guide,

Downloaded several DLLs but still have problem!

Do you mean OpenSSL for windows? You are not hyperlinked "here"!

The new error is this:

The ordinal 4445 could not be located in dynamic link library LIBEAY32.dll

Logged

sepehr

  • Hero Member
  • *****
  • Karma: +44/-6
  • Offline Offline
  • Posts: 810
  • CryptoCurrency Evangelist
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #17 on: May 27, 2014, 02:56:15 am »

Gave me this error:

"The program can't start because LIBEAY32.dll is missing from your computer. Try reinstalling the program to fix this problem"

Please guide
You need SSL on windows. Go here and download either the 32bit or 64bit version. You might need to download some of the C++ Redistributables to get it to work.

Then you may have problems with some other .dll files. msvcr110.dll or msvcr120.dll

You'll need to get them from here: http://www.dll-files.com/dllindex/dll-files.shtml?msvcr110

thanks a lot for your guide,

Downloaded several DLLs but still have problem!

Do you mean OpenSSL for windows? You are not hyperlinked "here"!

The new error is this:

The ordinal 4445 could not be located in dynamic link library LIBEAY32.dll

I have installed OpenSSL for windows and still same error happening.

Now I'm installing Microsoft Visual C++ 2010 Redistributable Package (x64) from here:

http://www.microsoft.com/en-us/download/details.aspx?id=14632

I hope the problem to solve, will update here for others to find the way
Logged

sepehr

  • Hero Member
  • *****
  • Karma: +44/-6
  • Offline Offline
  • Posts: 810
  • CryptoCurrency Evangelist
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #18 on: May 27, 2014, 02:58:53 am »

Gave me this error:

"The program can't start because LIBEAY32.dll is missing from your computer. Try reinstalling the program to fix this problem"

Please guide
You need SSL on windows. Go here and download either the 32bit or 64bit version. You might need to download some of the C++ Redistributables to get it to work.

Then you may have problems with some other .dll files. msvcr110.dll or msvcr120.dll

You'll need to get them from here: http://www.dll-files.com/dllindex/dll-files.shtml?msvcr110

thanks a lot for your guide,

Downloaded several DLLs but still have problem!

Do you mean OpenSSL for windows? You are not hyperlinked "here"!

The new error is this:

The ordinal 4445 could not be located in dynamic link library LIBEAY32.dll

I have installed OpenSSL for windows and still same error happening.

Now I'm installing Microsoft Visual C++ 2010 Redistributable Package (x64) from here:

http://www.microsoft.com/en-us/download/details.aspx?id=14632

I hope the problem to solve, will update here for others to find the way

oops! I had Microsoft Visual C++ 2010 Redistributable Package  in my computer!

and still the problem persists:

The ordinal 4445 could not be located in dynamic link library LIBEAY32.dll
Logged

sepehr

  • Hero Member
  • *****
  • Karma: +44/-6
  • Offline Offline
  • Posts: 810
  • CryptoCurrency Evangelist
    • View Profile
Re: NXTservices_div test release for dividend calculations at any block
« Reply #19 on: May 27, 2014, 04:15:19 am »

Sorry guys, I failed to run this script on windows yet!

Is anyone kind enough to run it on my assetID: 5934798443252900551 on Block: 146500 and give me the results?

Thanks
Logged
Pages: [1] 2 3 ... 7  All
 

elective-stereophonic
elective-stereophonic
assembly
assembly