Nxt Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Nxt Client: Nxt 1.11.14 - Latest Ardor Client: Ardor 2.0.14

Pages: [1]

Author Topic: Proposition for market driven variable fee in the Nxt ecosytem.  (Read 3446 times)

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

Hello fellow Nxter,

I write this proposition because I think the current fee for the prunable data part (i.e. Nxt/KB) is currently way too large and it should be market driven. Of course, there is always a fix fee that should be there to store the hash of the non prunable part of the transaction. What I'm discussing here is the fee for the prunable part of the transaction. However, It is very important to note that the methodology proposed here could be equally be apply to a base Nxt fee and to scale the rest of the Nxt ecosystem depending on that base fee. Overall, I think that testing this system for the prunable Nxt fee part might first be a good idea.

Note that the proposed expression for the fee take into account the value that the market think of what the fee should be.

I come up with an idea that I think is good and merit further discussion by the community. I think the idea is simple and elegant.
Overall, this idea is based on fluid mechanics formulas for which a fluid tends to relax toward a local equilibrium. However, the analogy here is that it is instead the Nxt fee that should relax toward the market driven announce value.

Let us first define a series of values which would be an announcement by each Nxt forger. Those values would represent the price at which they would ideally be willing to store prunable data (in Nxt/KB). Let take the median value of this series and name it fee_announce.

Let present the formula and then discuss it:
------------------
fee^{0} = 1 Nxt/KB
fee^{n+1} = fee^{n} - \omega * ( fee^{n} - fee_announce^{n} )
-------------
fee^{0} would be an initial fee to start the formula chain (it could instead be the current fee of NRS).
"n" would be the block number
\omega is a relaxation factor between 0 and 1.

If \omega is equal to zero, the fee do not change between block and this correspond to the current state of Nxt. To make a connection with fluid mechanic it would be like if the fluid is infinitely viscous and cannot relax toward an equilibrium.   

If \omega is equal to 1,  the fee automatically relax (in one block) toward the current block fee_announce^{n}. For a fluid analogy, this would be like the fluid instantly relax toward equilibrium. However, the announce value can change fast and be affect by various factors, I do not recommend such a value for \omega.

If 0 < \omega <1, the fee slowly or rapidly adjust itself to relax toward the current fee_announce^{n} value. This is guaranteed. Overall, \omega control the rate at which the relaxation occurs toward the equilibrium. To start with, \omega could equal to 10^-3 for example. But, it must be a small value to slowly relax the system toward the target equilibrium.

I made an computation example that show the result of this formula assuming fee^{0}=1, \omega=10^-3 and fee_announce{n}=0.5 for all "n".



We can see that the fee slowly relax toward the median fee that is announce by the forgers.

Please let know what you think.

I could make more computation if there is some interrogation.
« Last Edit: June 13, 2015, 05:22:16 pm by Sebastien256 »
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

nexTry

  • Newbie
  • *
  • Offline Offline
  • Posts: 7
    • View Profile
  • Karma: +1/-0

While 'lower fees' are always a good argument for using Nxt both as Unit of value as well as ecosystem, there is a problem.

Forgers get the fees for assembling blocks, not for storing them. While the processing of monetary transactions requires to have the past transactions in the database to verify the correctness of the transactions, prunable data has no dependency on older data. While it's possible to create a solution that enforces the storage of prunable data until the deadline is met. this  would produce probably unnecessary processing overhead.

Instead,this data should be paid by the consumer when receiving the data (via service subscription models). As a drawback, this would require some overhead since not only a hash but also the recipients adress had to be sored so users could evaluate the quality of the service.

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

While 'lower fees' are always a good argument for using Nxt both as Unit of value as well as ecosystem, there is a problem.

Forgers get the fees for assembling blocks, not for storing them. While the processing of monetary transactions requires to have the past transactions in the database to verify the correctness of the transactions, prunable data has no dependency on older data. While it's possible to create a solution that enforces the storage of prunable data until the deadline is met. this  would produce probably unnecessary processing overhead.

Instead,this data should be paid by the consumer when receiving the data (via service subscription models). As a drawback, this would require some overhead since not only a hash but also the recipients adress had to be sored so users could evaluate the quality of the service.

Thank for your input. But I'm not sure what the link between my post and your, but here i'm talking about market driven fee for forgers, which is presently how Nxt works. The proposition is valid for all fee in the Nxt network and aim at removing the burden of choosing the fee by the devs everytime with a hardfork. Instead, with this, the Nxt forgers adjust continuously the fee value instead of being manually adjust by the devs. The proposition is only about solving this problematic, nothing more.
« Last Edit: June 13, 2015, 04:14:41 pm by Sebastien256 »
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

superresistant

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 327
    • View Profile
  • Karma: +29/-3

 
I like this idea.
Just to know, what period of time between 1 NXT fees and 0.5 ?

TwinWinNerD

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2012
  • CEO BitPanda.com
    • View Profile
  • Karma: +222/-116

How do you create offline transactions with this system?

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24


I like this idea.
Just to know, what period of time between 1 NXT fees and 0.5 ?

Well, infinity in theory, the system always relax toward the announce value.  In pratice, however, in the example I gave it take about 5000 blocks to be very very close to 0.5. Overall, omega adjust the rate at which the system converge toward the target equilibrium, the number of block could be approximatly whatever the devs choses. Since the market target equilibrium would also change everyblock, the system will automatically adjust to slowly converge toward this new target value everyblock.

Market median fee annouce change at every block is like an external force acting on a fluid for which, thereafter, the fluid will always try to get in equilibrium state.
« Last Edit: June 13, 2015, 05:25:38 pm by Sebastien256 »
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

How do you create offline transactions with this system?

I guess that since fee are variable, fee in the transaction must be given at online broadcast, but I'm not sure at all. Overall, I'm no expert on this matter, but I think you could ask this same question for any variable fee system.
« Last Edit: June 13, 2015, 05:44:15 pm by Sebastien256 »
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

Riker

  • Core Dev
  • Hero Member
  • *****
  • Offline Offline
  • Posts: 1773
    • View Profile
  • Karma: +436/-42

How do you define the "market driven announce value" ?
Regarding relaxing the fee and the number of blocks in your formula, surely you cannot relax the fee for the upload prunable data transaction itself, since you already paid it when submitting it. Do you refer to the fee you need to pay in order to extend prunable data before it expires ?
NXT Core Dev
Account: NXT-HBFW-X8TE-WXPW-DZFAG
Public Key: D8311651 Key fingerprint: 0560 443B 035C EE08 0EC0  D2DD 275E 94A7 D831 1651

ChuckOne

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3450
  • ☕ NXT-4BTE-8Y4K-CDS2-6TB82
    • View Profile
  • Karma: +293/-17

How do you define the "market driven announce value" ?

I think he assumed an announcing system here and took the median of all announced values.

ChuckOne

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 3450
  • ☕ NXT-4BTE-8Y4K-CDS2-6TB82
    • View Profile
  • Karma: +293/-17

@Sebastian256

Interesting idea. It describes how Nxt can adjust the current fee to an agreed fee. In this case, we want a gradual change to the target fee.

We already pondered over the fee issue several times in the past. One of the main conclusions were the following requirements:

1) easy to compute and verify (performance)
2) no big changes (adaptability)
3) small number of non-zero digits (ease of reading)
4) caps on either side of the range (max + min)

The continuous approach (as your one is) sadly has a negative impact on the performance side whereas it has perfect properties for 2). 3) and 4) are still possible with some adjustments.

There is a much easier way to adjust fees and fee-related variables adhering to the above-mentioned requirements. I am going to reveal the idea in another thread when the time has come to improve Nxt fee-relatedly.

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

How do you define the "market driven announce value" ?
Regarding relaxing the fee and the number of blocks in your formula, surely you cannot relax the fee for the upload prunable data transaction itself, since you already paid it when submitting it. Do you refer to the fee you need to pay in order to extend prunable data before it expires ?

Hi lyaffe, TY for your questions.

As Chuckone said already, I took the median of all announce value that would be announce by forger nodes, but it could be someting else.
On your second question, I'm not so sure to understand it. I refer as the fee someone need to pay to the forger to process a transaction. What I propose is that this fee, instead of being fix in time (block-axis), be a continuous function always relaxing toward a target equilibrium value. The target equilibrium value would be, for example, the median of all announce value by the forger.

Concerning the prunable thing, I was refering to only test this methodology on this part of Nxt code first.
« Last Edit: June 14, 2015, 05:35:36 am by Sebastien256 »
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

@Chuckone,

Ok! I understand the approach I proposed might be difficult to verify in a p2p environment (performance side). I have no idea on how this work... If in anyway you required some of the idea here, let me know if you want some computational example test.
« Last Edit: June 14, 2015, 05:36:11 am by Sebastien256 »
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

nexTry

  • Newbie
  • *
  • Offline Offline
  • Posts: 7
    • View Profile
  • Karma: +1/-0

Well, if your only point is the way to decision making, my concerns are indeed off topic.

But your proposition could improved. Instead of the complex update process, which would require that each node gets each announcement (for every block) to ensure all  forgers compute the same fee. If there is a difference  nodes could discard transactions if their 'local' fee value is lower then the 'true' value. Another point is, you do not compute a average of the Fees, but the announcements. So trolls could set up some  forging nodes with a few Nxt, let them anounce nonsense fees.

Instead, i would propose to determine the market fee by the average/median/whatever of the fees of the last (7 *) 1440 blocks omitting the  blocks without fees. Allowed Fees for forgers would then have a lower bound of 0.6 ... 0.8 * this market average. Forgers who do not like such low fees  simple do not forge transactions below their treshold.

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

Well, if your only point is the way to decision making, my concerns are indeed off topic.

But your proposition could improved. Instead of the complex update process, which would require that each node gets each announcement (for every block) to ensure all  forgers compute the same fee. If there is a difference  nodes could discard transactions if their 'local' fee value is lower then the 'true' value. Another point is, you do not compute a average of the Fees, but the announcements. So trolls could set up some  forging nodes with a few Nxt, let them anounce nonsense fees.

Instead, i would propose to determine the market fee by the average/median/whatever of the fees of the last (7 *) 1440 blocks omitting the  blocks without fees. Allowed Fees for forgers would then have a lower bound of 0.6 ... 0.8 * this market average. Forgers who do not like such low fees  simple do not forge transactions below their treshold.

Hi,

I proposed median announcement value exactly to avoid troll nodes. It is possible that other value for the target equilibrium could be choose, it do not have to be the median of the announcement. It could be anything else suitable. You are right, however, that max, min fee range is also important to set up in the system, but, imo, these are technical details. Overall, devs would only needs to choose max, min and relaxation rate. Choosing only those 3 values would be simple and give much more liberty to what a free market should be concerning the fee.

Averaging the announcement value over mutiple block would has similiar effect as adjusting the relaxatíon rate omega. So the relaxion rate omega can be choose to take care of this, just take a smaller value if the overall changing rate is too large.

One weak point of the proposition is that it may be to much info to gather and process at every block. But maybe there is a way around this.
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

I'm reopening this topic and move a conversation from slack to here:

websioux [10:21 AM]
By the way, not really related: I miss one thing with Nxt / Ardor : How will the decision taken to modify the base transaction fee when it will be obvious it needs too? At 1$ we can start feel the need, but at 10$.. I know it can be a fatal choice, but not trying to clarify on it also puts us in a problematic centralized situation. If only we could know that we will have to debate and then vote. That acts as a kind of wall in my perception. (edited)

martis [10:44 AM]
Voting?
Websioux
By the way, not really related: I miss one thing with Nxt / Ardor : How will the decision taken to modify the base transaction fee when it will be obvious it needs too? At 1$ we can start feel the need, but at 10$.. I know it can be a fatal choice, but not trying to clarify on it also puts us in a problematic centralized situation. If only we could know that we will have to debate and then vote. That acts as a kind of wall in my perception.
Posted in #randomToday at 10:21 AM


sabretooth [11:18 AM]
I already suggested by voting. But you would have the client poll the results every time there is a transaction.

websioux [11:28 AM]
No not voting per transaction, not even automated, we do not know the best process. But once in a while, community should vote on reducing/increasing base fees. We all know that if NXT is at 10$, we have to reduce base fee, unless volume is already at max. I believe Nxters should be involved in this decision process. That's also true for others fees, some are linear with data size, which is quite fine, but 1000 Nxt for asset is quite arbitrary. The unknown about how this changes will be handle is our only weak point (for me).


sebastien256 [11:29 AM]
Would an announced system for the forger would work?
That way fee would be a free market.
I already had a proposition about this, but @ChuckOne said few year ago that the devs had a solution for that  (in my understanding).
you can check it out here :
and contribute also:
https://nxtforum.org/nxt-improvement-proposals/proposition-for-market-driven-variable-fee-in-the-nxt-ecosytem/msg183850/#msg183850
nxtforum.org
Proposition for market driven variable fee in the Nxt ecosytem.
 


websioux [12:29 PM]
If I understood your post was about a smooth transition toward an "announced" market fee. To avoid discontinuities. I wonder about establishment of that one. Forgers seems the most concerned to discuss about that. And in a way, we have the possibility to contact most of them. And since we can vote, then balance on stake, or better past forging efforts, pressure would be max for the devs to publish a new version. So in a way, I may consider that the fees of the futur will be decided by forgers.


sebastien256 [12:33 PM]
Voting would be certainly a nice solution too.
To be honest, I always wanted the devs to speak about their plan on this, but they never did to date yet. Would be nice if at some point in the near future this would be clarified @riker.

ChuckOne [5:01 PM]
@sebastien256 Unsolved issues always come back and say "hello my friend, remember me?"
If I remember correctly, those 4 points are still relevant:
1) easy to compute and verify (performance)
2) no big changes (adaptability)
3) small number of non-zero digits (ease of reading)
4) caps on either side of the range (max + min)

mrv777 [5:05 PM]
I do think caps are very important.  At least when first implimenting any dynamic fees, I would have a tight range

ChuckOne [5:05 PM]
Definitely.
Nxt was always about usability and readability. So, minimum fees should be like: 10, 1, 0.1, 0.01, 0.001 and so on. No need for more complicated stuff.
That also makes it easier to just shift the comma by 1 right or left when majority agrees on.
Once or twice a year should also suffice to cover the performance and adaptability point.
And that makes me think:
@sebastien256 Is it really necessary to do so automatically? Or is a Jelurida-run, stake-based voting enough to hard-code the new min fee in the official code?
I tend to make things easy and pragmatic. It min fees only change every 5 years, why would we need a sophisticated software solution + maintenance + potential risk of crashing the security of the network.

sebastien256 [5:42 PM]
@ChuckOne Thanks for reminding the points, I also think they are important and I'm not advocating my ~2 years old suggestions, here. I agree that simpler the better, but the things is that the potential solutions need to be discussed openly as we all know that crypto tokens are very volatile and if fees are unpredictable, projects would avoid block chain technology. I think Ardor childchains fees will be very sensitive to Ardor price variations and this is not good.

Most probably, stake-based voting would be useful and a simple solution. In fact, stake-based voting could be made at every X number of blocks automatically easily (or every block if it do not affect performance). A new transaction type could be added to Ardor, similar to the "account name", but it would be this time the "minimum fees" or something more appropriate announced by the forging account. This look like to me like a kind of announcing system that seem easy to implement. Something like this would avoid doing fork for changing fees. (edited)
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).

Sebastien256

  • Hero Member
  • *****
  • Offline Offline
  • Posts: 2818
  • ^LOOK UP^ = Nxt community!
    • View Profile
  • Karma: +169/-24

thomasveil [5:53 PM]
Forgers would prefer high fees, no? So a vote wouldn't lead to the best solution for everyone.


sebastien256 [5:54 PM]
No, forgers would not necessaly prefer higher fees. They would optimize the profit and profit goes up also with the number of transactions, which may required lower fees. (edited)

ChuckOne [5:55 PM]
@thomasveil Would they? They think a lot of how to protect their stake. So, killing the system by increasing the fees is not in their best interest.


thomasveil [5:56 PM]
I don't think they would kill the system. But they still might vote for higher fees than is optimal for users.


ChuckOne [5:56 PM]
What's optimal? :wink:

thomasveil [5:56 PM]
Look at miners - you would think they wanna have Bitcoin succeed - and yet a lot of them do stupid shit for short term profit.


[5:57]
Dunno. Unless the answer is "the highest fee possible", then it doesn't change what I said.


ChuckOne [5:57 PM]
Okay, so a hard max cap would be necessary.



[5:57]
It's like a social system.


[5:58]
Rich pay for the poor.


[5:58]
Forgers pay with their hardware for the users.


thomasveil [5:58 PM]
The hardware is not really a factor, is it?


ChuckOne [5:58 PM]
So, what else instead of Stake should be the way to measure the vote?


thomasveil [5:59 PM]
I didn't say there needs to be a vote.


sebastien256 [5:59 PM]
Impact of the account on the blockchain? I'm brainstorming here.


ChuckOne [6:00 PM]
Interesting idea, @sebastien256


thomasveil [6:00 PM]
Proof of Importance? :smile:


sebastien256 [6:01 PM]
lol this one I think is not available to the public.


ChuckOne [6:02 PM]
@thomasveil we can call it whatever we want it. voting, negotiating, crying, polling.... in the end some developers will code it. And the question is: whom will they ask? and how important is each response to them.


[6:02]
:smile:


thomasveil [6:03 PM]
True.


sebastien256 [6:04 PM]
I think questioning the users (people that make transaction) and questionning the forgers (people that pay for the system) is important


ChuckOne [6:04 PM]
Let @sebastien256 decide. He thought so much about these fees :smile:


sebastien256 [6:04 PM]
haha not that much @websioux started it all over :slightly_smiling_face:


thomasveil [6:04 PM]
So far just the devs decided. And it worked out fine.


[6:04]
But of course, they can be "fired" by the users and forgers if need be.


sebastien256 [6:05 PM]
They can't with JPL.


thomasveil [6:05 PM]
By not adopting their fork.


sebastien256 [6:05 PM]
Well that is true, but that is the nodes that decide I think.


ChuckOne [6:06 PM]
Forging nodes decide.


[6:07]
I go and grab some sleep. Cya guys.


sebastien256 [6:07 PM]
Sleep well.


thomasveil [6:07 PM]
n8 :slightly_smiling_face:


sebastien256 [6:10 PM]
The big problem by choosing the best fee for the platform is that its a multivariate optimization problem (users, forgers, etc). These multivariate optimization are very difficult since afaik it always required human intervention to choose the best points on the pareto front. (edited)


[6:11]
I don't know if some people are understanding what i'm saying....


mrv777 [6:12 PM]
You could add two things too:
The fee options are only stay the same, increase fees to next power of ten, or decrease fees to the next power of ten. Simple.
Also, base the vote off of account balance but cap the weight to the average of all accounts voting. To give a little balance against whales (edited)


sebastien256 [6:13 PM]
In such a case, only forger are solicit. This is one-dimensional optimization and the easiest solution.


mrv777 [6:15 PM]
I would just vote with a flag in the conf file too


[6:15]
Well any user can start a forging node around the voting periods


[6:16]
Or have the option to vote in the client too


[6:17]
Adds complexity but shows off one of the features we have. Using our own voting system to decide fees


[6:17]
I would think that would be harder to code though


[6:18]
I think just a flag in conf for forgers is a good initial step. With just three options, increase, decrease, stay the same (edited)


[6:18]
I think we did do a poll about fees for NXT a long time ago


sebastien256 [6:19 PM]
Could be a simple solution. I would really love to see input from Jelurida on their view on the subject so.
Please drop your ideas concerning Nxt and/or NRS in this topic -> List of feature request for Nxt and/or NRS (with the full list in OP).
Pages: [1]