Topic: Adapting client-side code to the fee changes planned for 1.7 (Read 2988 times)

Jean-Luc · « **on:** November 11, 2015, 05:18:30 pm »

This topic is for technical details on how a migration to dynamic, server-side calculated fees can be achieved.

Regardless of the exact fee values we set to take effect after the 1.7 hardfork, there will always be a need to adjust the fees in response to market conditions, so keeping a hardcoded fee of 1 NXT for most transaction types is no longer a viable approach for client applications. But hardcoding the logic to calculate minimum fees on the client side, and keeping it up to date with this and future fee changes is also not maintainable. Fortunately, we have the feature that submitting a transaction with feeNQT=0 will have the server calculate the minimum fee needed automatically, and this feature is alredy in production in the 1.5 and 1.6 branches. This works for transactions being signed on the server only, as once the transaction bytes are signed, it is no longer possible to change any of them, including the fee. But for client-side signing, the client can first submit the transaction with feeNQT=0, broadcast=false, and publicKey instead of secretPhrase, and allow the server to calculate the required fee and prepare the unsigned transaction, to be signed client-side after possibly confirming the fee with the user. To minimize the impact of fee changes in 1.7 and avoid having to hardcode any fee logic in the clients, developers should start modifying their code to take advantage of this method of fee calculation, without waiting for the first 1.7.0e release. A migration to prunable messages (achived by adding messageIsPrunable=true parameter when submitting the transaction) must also be started, without waiting for 1.7, and when using a permanent message is prefered and the user is willing to pay the higher fee, the future limit of 160 bytes should be kept in mind, possibly starting to enforce it in client-side applications even now.

To make the migration to dynamic server-side fee calculation even easier, we plan to add a configuration parameter in nxt.properties. When enabled (default will be disabled), the server will ignore invalid (lower than the minimum) fees submitted by the users, and will replace those with the minimum required fee, as if the user has specified a fee of 0. This can of course only work for unsigned transactions, if the transaction is already signed on the client side with an insufficient fee, that fee can no longer be changed by the server.

Jean-Luc · « **Reply #1 on:** November 17, 2015, 05:37:52 pm »

In case anyone did not pay attention:

1. There will be changes in fees and permanent message size limit in 1.7. They will take effect in the 1.7 hard fork, in very early January 2016.

2. These changes will break any application that has hardcoded fees of 1 NXT, or uses permanent messages exceeding 160 bytes.

3. THESE CHANGES HAVE BEEN ANNOUNCED ALREADY, AND ALL API USERS CAN, AND ABSOLUTELY MUST START UPDATING THEIR CODE NOW. DO NOT WAIT UNTIL 1.7 STABLE IS RELEASED, AND THEN COMPLAIN THAT YOU HAVE NOT BEEN GIVEN ENOUGH NOTICE. START UPDATING TO 1.6.2, USE SERVER-SIDE FEE CALCULATION, AND EITHER SWITCH TO PRUNABLE MESSAGES, OR MAKE SURE PERMANENT MESSAGE SIZE DOES NOT EXCEED 160 BYTES.

4. To repeat. Do not wait until 1.7.1 stable, and then blame the developers for not giving sufficient advance notice once it is suddenly released and the hard fork is right around the corner. THIS IS THE ADVANCE NOTICE.

altsheets · « **Reply #2 on:** November 17, 2015, 06:21:17 pm »

Thanks for the warning.
Man, there must be drama behind this, judging by the tone

.

I just had an idea, to possibly minimize the impact of this upcoming novelty.

When such fundamental, app-breaking innovations are necessary,
everyone has to solve the same question at the same time. Right?

So ... idea: A centralized approach.

Publish example code, or whole libraries, in all major languages;
solving this (and that) same-for-everyone challenge. Once.

An example:
Many apps had not have to do that "broadcast=false" round-trip yet.
If there was a ready library that everyone can plug into ... = easier.

That is not much work, but if there are no resources for that right now, still
it would be more efficient if it was done only once, not in hundreds of apps.
A new asset? Crowdfund nxt API libraries, in Python, JS, Java, PHP, Ruby, C.

Or at least in one language, ideally the best readable.
Then translations into other languages are really easy.

And if in the future, more API changes need to be made,
updating that abstracted library might already do the trick.
--> An abstraction layer for the nxt API.

Meant constructively, not criticism. Just sharing an idea ...

Jean-Luc · « **Reply #3 on:** November 17, 2015, 07:33:27 pm »

Quote from: altsheets on November 17, 2015, 06:21:17 pm

I just had an idea, to possibly minimize the impact of this upcoming novelty.

When such fundamental, app-breaking innovations are necessary,
everyone has to solve the same question at the same time. Right?

So ... idea: A centralized approach.

Publish example code, or whole libraries, in all major languages;
solving this (and that) same-for-everyone challenge. Once.

An example:
Many apps had not have to do that "broadcast=false" round-trip yet.
If there was a ready library that everyone can plug into ... = easier.

That is not much work, but if there are no resources for that right now, still
it would be more efficient if it was done only once, not in hundreds of apps.
A new asset? Crowdfund nxt API libraries, in Python, JS, Java, PHP, Ruby, C.

Or at least in one language, ideally the best readable.
Then translations into other languages are really easy.

And if in the future, more API changes need to be made,
updating that abstracted library might already do the trick.
--> An abstraction layer for the nxt API.

Meant constructively, not criticism. Just sharing an idea ...

If your application uses API calls that create new transactions, and submits the secretPhrase to the server, the only change is to remove any hardcoded feeNQT=100000000 parameters to those calls, and replace with feeNQT=0. Then the server will calculate and use the minimum fee needed automatically.

The broadcast=false method should only be used if your application does client-side signing of transactions, creating the transaction client-side and only broadcasting it later.

altsheets · « **Reply #4 on:** November 18, 2015, 05:01:47 pm »

Thanks!

Quote from: Jean-Luc on November 17, 2015, 07:33:27 pm

If your application uses API calls that create new transactions, and submits the secretPhrase to the server, the only change is to remove any hardcoded feeNQT=100000000 parameters to those calls, and replace with feeNQT=0. Then the server will calculate and use the minimum fee needed automatically.
The broadcast=false method should only be used if your application does client-side signing of transactions, creating the transaction client-side and only broadcasting it later.

Thanks for the clarification. Makes sense.

And what do you think about this idea, wrapping the whole API into a module,
that over time more and more on-top-of-NXT developers can switch too?

Quote from: altsheets on November 17, 2015, 06:21:17 pm

...everyone has to solve the same question at the same time. Right?
So ... idea: A centralized approach.
Publish example code, or whole libraries, in all major languages;
solving this (and that) same-for-everyone challenge. Once.
...
That is not much work, but if there are no resources for that right now, still
it would be more efficient if it was done only once, not in hundreds of apps.
A new asset? Crowdfund nxt API libraries, in Python, JS, Java, PHP, Ruby, C.
Or at least in one language, ideally the best readable.
...
And if in the future, more API changes need to be made,
updating that abstracted library might already do the trick.
--> An abstraction layer for the nxt API.

As a side effect, such a module would make app development for nxt even easier.
--> The NXT ecosystem grows.

Devs can fully focus on their app. They rely on a -centrally organized,
and even code reviewed- and simple-to-use module, in their language.

I have seen examples for (admittedly simpler) APIs at poloniex (PHP and Python wrappers,
and node.js example) and bittrex (go, ruby, node, python wrappers and example code).

I have now simply opened a github repo. Not sure yet how to organize it exactly.
But it exists now:
https://github.com/altsheets/nxtwrap

Everyone who wants to submit his own draft for NXT api wrappers,
please just upload your code. I imagine in the beginning there are several
approaches, and over time they could converge. All languages welcome!

Or are there already other, existing repos like this? Then I remove it again.

blackyblack1 · « **Reply #5 on:** November 18, 2015, 05:35:56 pm »

Quote from: Jean-Luc on November 17, 2015, 07:33:27 pm

The broadcast=false method should only be used if your application does client-side signing of transactions, creating the transaction client-side and only broadcasting it later.

Sorry I'm a bit lost here. How to create a transaction for calculating the fee without transferring secret phrase to server which is the idea of client-side signing.

Jean-Luc · « **Reply #6 on:** November 18, 2015, 05:48:10 pm »

You need to supply a publicKey parameter, with the public key of the sender, instead of secretPhrase. Then you don't even need to add broadcast=false, if there is no secretPhrase the transaction cannot be signed and is therefore not broadcasted. You get back the (unsigned) transactionJSON and unsignedTransactionBytes, and can check what the fee has been set to in each of them.

Switch to ArdorForum

News:

Author Topic: Adapting client-side code to the fee changes planned for 1.7 (Read 2988 times)

Jean-Luc

Adapting client-side code to the fee changes planned for 1.7

Jean-Luc

Re: Adapting client-side code to the fee changes planned for 1.7

altsheets

Re: Adapting client-side code to the fee changes planned for 1.7

Jean-Luc

Re: Adapting client-side code to the fee changes planned for 1.7

altsheets

Re: Adapting client-side code to the fee changes planned for 1.7

blackyblack1

Re: Adapting client-side code to the fee changes planned for 1.7

Jean-Luc

Re: Adapting client-side code to the fee changes planned for 1.7