Nxt Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Nxt Client 1.11.10 - NEW RELEASE: Ardor 2.0.5e TestNet - The Ignis ICO is over!! Ardor genesis snapshots will happen at Nxt block 1,630,000 (expected for 25th December)

Pages: 1 [2] 3  All

Author Topic: Nxt Energy and Cost Efficiency paper update.  (Read 3961 times)

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

I see what you're getting at with HF, PC was proposed as a way of allowing a country to have their own branch and people locally transact on that branch.  They can however move their Nxt from one branch to another.  I'm not sure I fully understand your HF proposal.. maybe this weekend I can read your forum topic a little bit closer.

I guess how strongly we need to optimize largely depends on how many transactions we can handle at the moment.  We're talking about somewhat reworking the transparent forging code, so you want to discuss optimizations, I'd love to hear them.

I agree that it would be very nice to have someone analyze the exact numbers and set-up some tests of how long it takes Nxt to verify transactions, package then into a block, sign it and send it out to the network.  As well as how long it takes to unpack it and verify.  I believe that the most expensive operations are the Curve25519 signatures and verification, which is why it's going to be very nice when something like this that can handle 32,000 Curve25519 operations per second is released: http://www.hgi.rub.de/media/sh/veroeffentlichungen/2014/03/25/paper_arc14_curve25519.pdf  and if needed could even be parallelized further.

However, I would indeed like to analyze the bandwidth requirements slightly closer... just wish I had more time to spend on Nxt.  Can't wait until the price jumps to $1 or so, would be nice to quit my job and throw myself full time into Nxt development.
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

chanc3r

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 1019
  • NXTInspect
    • View Profile
  • Karma: +124/-50


Back on topic, finally have a little bit of time.  It's not a core part of the paper, I could even drop it or just make a small note that at Bitcoin's size we'll only need to cover 1 transaction per second and can easily handle that. But sitting down now to try to work on this and come up with my own numbers.  Bob_ggg, you want to bounce around numbers, let me know.

Guys - love the discussion but please finish the paper - then if you want go on an optimise NXTs network requirements in the next paper please.... :)
NXT: 29996814460165 (NXT-JTA7-B2QR-8BFC-2V222)
@imrimr @NXTinspect

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

Good point :)

How important do you guys feel that the bandwidth calculation is?  Could I simply remove it from the paper?

As I see it, it's the last thing holding this up.  Anyway still working on that big project at work.. I will try to do this bandwidth calculation asap.
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

bob_ggg

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 90
    • View Profile
  • Karma: +9/-0

An approximate solution could use the fact that the forging node needs Txn data (current block), the previous block and has to send the forged block to the next forger. Worst cases for each of these situations are known. Add to this time the computing time and you get an estimate of the throughput.

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

Well computing time is minimal, the next forger in line should be collecting transactions, when the new chain comes, he quickly runs through all the transaction he has and checks if they've been included. He now has a list of transactions that have not been included, he starts packaging them together and I believe he doesn't need to create the entire block before hashing transactions as they come in.  So when it's time up for him, he signs it, which is one single Curve 25519 operation, and sends it to the next forger in line and the process repeats.  Point being, I think a negligible amount of processing takes place that can't be done within the minute that the forger is forging.  So if we're discussing bandwidth alone, I think it would be time it takes to send the current block the the next forger is the biggest issue..  which is exactly what I already have.

Why do we need to worry about anything more than that? I think the paper is done as is, though that could be better explained within there.
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

chanc3r

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 1019
  • NXTInspect
    • View Profile
  • Karma: +124/-50

Well computing time is minimal, the next forger in line should be collecting transactions, when the new chain comes, he quickly runs through all the transaction he has and checks if they've been included. He now has a list of transactions that have not been included, he starts packaging them together and I believe he doesn't need to create the entire block before hashing transactions as they come in.  So when it's time up for him, he signs it, which is one single Curve 25519 operation, and sends it to the next forger in line and the process repeats.  Point being, I think a negligible amount of processing takes place that can't be done within the minute that the forger is forging.  So if we're discussing bandwidth alone, I think it would be time it takes to send the current block the the next forger is the biggest issue..  which is exactly what I already have.

Why do we need to worry about anything more than that? I think the paper is done as is, though that could be better explained within there.

I would just clean it up a bit and publish....
NXT network bandwidth is something that will need further work in the future and for one would like to see inf-com looking for people to propose solutions for...
NXT: 29996814460165 (NXT-JTA7-B2QR-8BFC-2V222)
@imrimr @NXTinspect

bob_ggg

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 90
    • View Profile
  • Karma: +9/-0

Well computing time is minimal, the next forger in line should be collecting transactions, when the new chain comes, he quickly runs through all the transaction he has and checks if they've been included. He now has a list of transactions that have not been included, he starts packaging them together and I believe he doesn't need to create the entire block before hashing transactions as they come in.  So when it's time up for him, he signs it, which is one single Curve 25519 operation, and sends it to the next forger in line and the process repeats.  Point being, I think a negligible amount of processing takes place that can't be done within the minute that the forger is forging.  So if we're discussing bandwidth alone, I think it would be time it takes to send the current block the the next forger is the biggest issue..  which is exactly what I already have.

Why do we need to worry about anything more than that? I think the paper is done as is, though that could be better explained within there.
I agree that with the current transaction rate, these issues are not that relevant. Looking forward to a platform that supports transactions at VISA-like rate, they are.
Just a couple of questions regarding your statements:
  • my understanding is that the "next" forger needs the current block to make sure that transactions are not counted two times to avoid double spending; if this is the case, block assembly can start after this verification step is complete. What the latency introduced by this step?
  • the Txn rate is an element that represents a key advantage wrt other currencies such  as BTC. IMO, it would be worth spending some time in the next release of the report dealing with the possibility to reach instant response from the system while remaining on-chain. The one minute time between two blocks would be too long for a customer waiting in line for a confirmation.
In any case, this paper is a very good read.

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

What I'm saying is that before we consider instant transactions (which I agree is important!), the expensive part of the processing is signing transactions, the forger can be signing transactions as they come in during the block before he forges.  Then the expensive part is checking whether or not he should include them when they come in.  Which yes, takes time but isn't a particularly expensive operation.

So you are right, and let me see if I can run some quick math assuming the two can not be parallelized easily.

I've appreciated all the feedback and discussion bob_ggg!
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

Ok, finally updated it, just tweaked the wording and said that the bandwidth requirements would increase with instant transactions and I think it's ready to go.  You guys say the word and I'll send this to Damelon to publish and then send it to Coindesk, spread it around the nxt forums, bitcoin forums etc, and try to get this article out into the eyes of the general public.  Unless for any reason you think we should wait longer before publishing these numbers?
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

Bump.. maybe if I post a new reply this will show up as having been updated and people will read this.

Anyway I think the paper is done.. I tweaked the introduction a little bit to make Bitcoin out as a little bit more of a peer competitor, and updated to the newest numbers as Bitcoin just jumped up to a hashing power of 77,000,000 GH/s today.. and doesn't look like it will stop going up anytime soon.. just continuing to improve these numbers.

Haven't heard from SecondLeo in a very long time..  anyway waiting for someone to approve it.
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

could have Nxt similar graph?
https://blockchain.info/charts/cost-per-transaction

I agree that would be a nice one to have and would really show off Nxt's power.. though at the moment, not 100% sure it would be much better than Bitcoin's graph.
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

bob_ggg

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 90
    • View Profile
  • Karma: +9/-0

Reading again the paper, I suggest to introduce an explanation of the number of nodes that are required for forging Nxt blocks. The paper suggests that the number of Nxt nodes should be the same of that used by BTC. The reasoning is unclear (at least to me). What is the advantage of having a number of nodes larger than a few dozens now that leasing is supported?
Maybe I am misunderstanding this point, but if Nxt really needs a smaller number of nodes, the cost of a transaction would be more in line with (my) expectations.

chanc3r

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 1019
  • NXTInspect
    • View Profile
  • Karma: +124/-50

Reading again the paper, I suggest to introduce an explanation of the number of nodes that are required for forging Nxt blocks. The paper suggests that the number of Nxt nodes should be the same of that used by BTC. The reasoning is unclear (at least to me). What is the advantage of having a number of nodes larger than a few dozens now that leasing is supported?
Maybe I am misunderstanding this point, but if Nxt really needs a smaller number of nodes, the cost of a transaction would be more in line with (my) expectations.

This approach was taken to show the energy difference between NXT and Bitcoin without over favouring NXT I think, so its to make the comparison easier, of course NXT would need far fewer nodes but the knowledge to calculate exactly how many nodes are needed seems to escape us so it is very over specified but the energy difference is still staggering.
NXT: 29996814460165 (NXT-JTA7-B2QR-8BFC-2V222)
@imrimr @NXTinspect

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

A lot of people feel like the more nodes, the more decentralized, the better.  I wasn't sure how to estimate it but figured that probably that would be a reasonable estimate.

I suspect Nxt will be lower than that because it'll only be giving people transaction fees instead of 25 BTC per block, but I figure that even being generous to Bitcoin like this,

If you have some reasoning that leads to a number, I would love to hear it.

Maybe 2 cents per transaction, leads to 8 cents per block at Bitcoin's size as opposed to $10,000 per block as Bitcoin has.. maybe that number could be used to reduce the number of forgers.  People won't be getting much off of supporting the Nxt network as they hope to from supporting the Bitcoin network.. which leads to significantly fewer forgers.  Now that I think about it.. it probably would be significantly lower number of nodes.

If you can come up with a way to quantify that, I'd love to hear it.  Thanks bob_ggg!
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

bob_ggg

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 90
    • View Profile
  • Karma: +9/-0

Not easy, but here is an attempt.....

First of all, PoS + lending leads to centralization (same as PoW) unless there are reasons against this trend.
These reasons should be priced and currently they are not. (For instance, there is no currently implemented upper bound on the amount of coins that a pool can hold.)
Hence, the main difference between PoW and PoS is that the first one requires competition to get the best computing resources while the other increases competition among pools to get the largest amount of coins. In fact, if a pool of forgers rewards the lenders with a fraction of the collected fees, given that the cost of infrastructure is largely a constant, the economic efficiency of such a partnership increases with its size.
Following this line of reasoning, a rational player would allocate the coin to a pool with the ability to stay reliably on line to avoid penalization that would lower the chances to forge and the rewards paid to the lenders.

Given this landscape, it seems to me that a CPU as cheap as possible given the upper bound of Txn/sec derived in the paper and located in the most protected environment (e.g. with the usual support of reliable power supply, protection against DDoS attacks and so on....) would be the most obvious choice.
The number of these CPU is almost irrelevant given your analysis which would actually provide a strong support for a totally centralized server running somewhere. In any case, avoiding this extreme conclusion, it seems that a few dozen CPU distributed in many different locations are plenty to support the foreseeable computing load.

If we accept this line of reasoning, the estimated number of CPUs required to support forging would decrease from the 8.000 in the paper to about 100. This lower number is independent of the fees payed and based only on the numbers you provided in the paper. The ultimate conclusion is that you get the following comparison between BTC and NXT:
  • number of relevant pools of miners in BTC: about 20; number of pools of forgers in NXT: about 100
  • number of CPUs associated to each NXT pool: 1; required computing resorces by BTC pools: on average several petahashes/sec divided by the number of pools of miners
  • relative energy cost of Nxt vs. BTC: your numbers reduced by about 80 times (from 8.000 to 100).

The lingering question is if the community likes this trend toward centralization, but this is a different question.

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

Not easy, but here is an attempt.....

First of all, PoS + lending leads to centralization (same as PoW) unless there are reasons against this trend.
These reasons should be priced and currently they are not. (For instance, there is no currently implemented upper bound on the amount of coins that a pool can hold.)
Hence, the main difference between PoW and PoS is that the first one requires competition to get the best computing resources while the other increases competition among pools to get the largest amount of coins. In fact, if a pool of forgers rewards the lenders with a fraction of the collected fees, given that the cost of infrastructure is largely a constant, the economic efficiency of such a partnership increases with its size.
Following this line of reasoning, a rational player would allocate the coin to a pool with the ability to stay reliably on line to avoid penalization that would lower the chances to forge and the rewards paid to the lenders.

Given this landscape, it seems to me that a CPU as cheap as possible given the upper bound of Txn/sec derived in the paper and located in the most protected environment (e.g. with the usual support of reliable power supply, protection against DDoS attacks and so on....) would be the most obvious choice.
The number of these CPU is almost irrelevant given your analysis which would actually provide a strong support for a totally centralized server running somewhere. In any case, avoiding this extreme conclusion, it seems that a few dozen CPU distributed in many different locations are plenty to support the foreseeable computing load.

If we accept this line of reasoning, the estimated number of CPUs required to support forging would decrease from the 8.000 in the paper to about 100. This lower number is independent of the fees payed and based only on the numbers you provided in the paper. The ultimate conclusion is that you get the following comparison between BTC and NXT:
  • number of relevant pools of miners in BTC: about 20; number of pools of forgers in NXT: about 100
  • number of CPUs associated to each NXT pool: 1; required computing resources by BTC pools: on average several petahashes/sec divided by the number of pools of miners
  • relative energy cost of Nxt vs. BTC: your numbers reduced by about 80 times (from 8.000 to 100).

The lingering question is if the community likes this trend toward centralization, but this is a different question.

Actually there has been talk about putting a cap on the max number of Nxt you are allowed to forge with within a pool.  More people than this can contribute but it'll behave as if it had this limit on the number of Nxt per pool.  We were bouncing around anywhere from 1 to 10 million, though we'll need to get community input.

Say we go with 1 million though, that leads to 1000 pools, which is definitely a better number from both the distribution as well as energy efficiency number.  Question is, how many solo forgers will there be who don't meet this cap?

We would also like a minimum cap to prevent anyone with lots of Nxt from splitting them across accounts and sybil attacking with lots of tiny accounts.  We're thinking something like 100,000 Nxt per forging pool limit on this end.

Keep in mind the opposite force that would lead people to solo forge a little bit more is that it is far cheaper to forge on your PC than to mine for Bitcoin.  You do still need to load up on Nxt but if you have them, might as well mine.  I think that would be outweighed by the want to mine 24/7 and the hassle and I think most people would trust a mining pool instead but not 100% of people.

I mean I think I can logically argue that we'll have less people forging due to the decreased payouts alone.

Would everyone be ok with 1000 forging pools?
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

bob_ggg

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 90
    • View Profile
  • Karma: +9/-0

Economic efficiency would push the forging pools to reach their maximum size, so if the community introduces a cap @1M, then 1000 would be in line with a rational behavior of the NXT holders.

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

If that is the natural progression, why does Bitcoin have so many individual nodes?
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q

bob_ggg

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 90
    • View Profile
  • Karma: +9/-0

Each mining node can be part of a pool increasing its chances to win the block. The model is that each node must be active to have a chance to win.
In the Nxt case, there is no advantage in having more nodes than those strictly required by the computng requirements you analyzed or by other bounds set by the community. As you pointed out in the paper, the nodes not forging remain idle. This is very different from the BTC case.

mczarnek

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 898
    • View Profile
    • Nxt Place - Craigslist for Nxt
  • Karma: +68/-4

Certainly I suspect that low return's on investment will not lead to people rushing to forge in the same way.

I suspect that it might be slightly higher than that though because people trust themselves more than anyone else.  So there probably will be those who feel like if I forge with my 10,000 Nxt then that is the only way I can be sure that my 10,000 Nxt is not being used by someone trying to attack the network.

hmm.. maybe I should ask this question to the community in general and see what the average person thinks.. the way they think may change that number.

I do somewhat see what you are saying, if you have a Bitcoin miner, you are powering it and connecting it to the network one way or another.. it's just a matter of whether you join a pool or mine solo.  If you are forging, then that's less connected to the coin.

I suspect we will definitely have some lone forgers though, who forge with significantly less Nxt than the maximum cap.
NXT Organization: Tech
Donations greatly appreciated: NXT-DWVJ-G89C-RHNL-6QW6Q
Pages: 1 [2] 3  All