elective-stereophonic
elective-stereophonic
NRS v1.5.1e  
Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Stable Nxt Client: Nxt 1.12.2

Pages: 1 2 3 [4]  All

Author Topic: NRS v1.5.1e  (Read 22639 times)

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.1e
« Reply #60 on: April 14, 2015, 04:18:18 pm »

anyway a node can disable pruning for specific destinations/senders?
with a default that a prunable message sent by a user's node wont disappear from their node.
Not currently, but it will be simple to add. The prunable_message table has sender and recipient columns, only need to enable filtering on those when the pruning is run.
Quote
Now if they ever need to redownload the blockchain, they would have to get these pruned messages unpruned from archival servers. Will there be a mechanism to retrieve such things.
I added a property to force always to include the prunable parts in the transaction json. If enabled on such an archival server, a node that downloads the blockchain from it will also receive the pruned parts.
There is also the getAllPrunableMessages API to get them as json, but currently no API to store that json back in the database, well, somebody can write such a tool.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.1e
« Reply #61 on: April 14, 2015, 04:30:35 pm »

Any of my own internal transactions that changes the core internal state are stored in the AM (average 10 bytes), and any large arbitrary data is stored as a 32 byte hash and distributed outside of Nxt.

If I am forced to use prunable messages then I can no longer store state changes (at most 10-20 bytes) in the blockchain and can only use hashes.. which means the user will reconstruct state randomly as content chunks come it, and if a content chunk does go missing, you cant guarantee the apps internal state is matching everyone else.
If what you need is only 10-20 bytes, or up to 100 bytes, this will still be available as a permanent AM for no extra fee. My proposal is to reduce the free limit from 1000 bytes to something like 100 bytes, and have extra fee for up to 1000 bytes, not to completely eliminate the permanent AMs.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.1e
« Reply #62 on: April 14, 2015, 05:07:15 pm »

You are buying ultra long storage space, available everywhere, and without attack point (unless the entire blockchain is lost). Using the Cloud, we know that important data could be compromised; with this, it is a thing of the past. For this price, you can easily store encrypted your most important personal things, you just have to wait for an easy app to do this (written in Jay ?).

We should stop thinking about the blockchain as storage space. When posting data to the blockchain (that are not related to its own functioning), what it should provide is distribution (after achieving the consensus on which transactions/data are accepted), and trustless future verification of this data. The storage part of it should shift to service providers.

Some nodes can opt in to keep prunable data very long, even indefinitely. But the protocol should not force all the nodes to do it.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

jones

  • Hero Member
  • *****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Re: NRS v1.5.1e
« Reply #63 on: April 14, 2015, 05:16:45 pm »

You are buying ultra long storage spaaace, available everywhere, and without attack point (unless the entire blockchain is lost). Using my Butt, we know that important data could be compromised; with this, it is a thing of the past. For this price, you can easily store encrypted your most important personal things, you just have to wait for an easy app to do this (written in Jay ?).

We should stop thinking about the blockchain as storage spaaace. When posting data to the blockchain (that are not related to its own functioning), what it should provide is distribution (after achieving the consensus on which transactions/data are accepted), and trustless future verification of this data. The storage part of it should shift to service providers.

Some nodes can opt in to keep prunable data very long, even indefinitely. But the protocol should not force all the nodes to do it.

We can set up a full storage node, so that all prunable data will be kept in at least one place, that way you can write applications that store extremely large amounts of data, then you can get the data from a node, (lets say a call to jnxt) and you can compare the data to the hash on your system to validate it.

This way nodes only have to store the 32 bytes, and AM can be of any length they want, some slight adjustments to code could make this emulate old AM (but without the 1000 byte limit) fairly easily.
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.1e
« Reply #64 on: April 14, 2015, 09:35:51 pm »

Quote
Why do you think that prunnable parts are "by definition" never parts of the transaction? Prunnable parts are never signed but they can be included in the transaction safely. When they are prunned the sign still will be valid but they will be removed from the transaction.
They are a part of the transaction, but not of the transaction bytes. Only the hash is a part of the bytes.
What is the problem moving them to the bytes? They are not signed so can be easily removed from the bytes.
We cannot add and maintain yet another bytes format, which is not used anywhere in the core. We already support json for the complete transactions, and bytes for the signed part only. To be able to submit the prunable parts to broadcastTransactions, there will be two ways: either use the transactionJSON parameter, which includes everything (and if doing the signing client side, just put the signature in it too), or use the transactionBytes for the signed bytes and put the prunable parts in prunableAttachmentJSON, with the same format as the transaction.attachment JSON (non-prunable parts in it will be ignored) which is returned by all CreateTransaction APIs.

Adding separate message, encryptedMessage, and so on parameters to broadcastTransaction will indeed not scale as we need to add more and more prunable parts, this is why we need to replace them with that single prunableAttachmentJSON parameter in 1.5.2e.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

SwissAlps

  • Hero Member
  • *****
  • Karma: +31/-16
  • Offline Offline
  • Posts: 519
    • View Profile
    • NxtTracker.com
Re: NRS v1.5.1e
« Reply #65 on: April 15, 2015, 05:23:14 am »

You are buying ultra long storage spaaace, available everywhere, and without attack point (unless the entire blockchain is lost). Using my Butt, we know that important data could be compromised; with this, it is a thing of the past. For this price, you can easily store encrypted your most important personal things, you just have to wait for an easy app to do this (written in Jay ?).

We should stop thinking about the blockchain as storage spaaace. When posting data to the blockchain (that are not related to its own functioning), what it should provide is distribution (after achieving the consensus on which transactions/data are accepted), and trustless future verification of this data. The storage part of it should shift to service providers.

Some nodes can opt in to keep prunable data very long, even indefinitely. But the protocol should not force all the nodes to do it.

We can set up a full storage node, so that all prunable data will be kept in at least one place, that way you can write applications that store extremely large amounts of data, then you can get the data from a node, (lets say a call to jnxt) and you can compare the data to the hash on your system to validate it.

This way nodes only have to store the 32 bytes, and AM can be of any length they want, some slight adjustments to code could make this emulate old AM (but without the 1000 byte limit) fairly easily.

It is true that this kind of data should not stay on the main blockchain, but only their hashes, I agree 100 % with that; but it is also quite obvious that a lot of people need an easy way to store data on a peer-to-peer and inviolate base, and it would be nice to be able to use the Nxt environment to do that  :)

But may be it would be possible to have a child blockchain that is made entirely of 1MB encrypted blocks, whose Ids is their hashes. Nobody needs to store this entire blockchain because the main blockchain keeps the verified hashes.

If a client uses one of these blocks, he pays for this use by storing 30 blocks on his machine.
 
The blocks still belongs to an Nxt account, and the current AM mechanism could work, when sending one of this block to another acount, we would send only the hash.

We could use the following features
  • CreateMegaBlock, this creates a block of 1MB of encrypted data that belongs exclusively to the Nxt account, cost could be 50 NXT
  • UpdateMegaBlock, replacing the contents by a new one, pay could be 5 NXT
  • DeleteMegaBlock, pay 1 NXT
  • RetrieveMegaBlock, it is free but the caller has to store 30 blocks for 1 he uses
Logged
CryptoNanoPay project
Note that the "Barter Point" test has just started...

blackyblack1

  • Hero Member
  • *****
  • Karma: +165/-82
  • Offline Offline
  • Posts: 1764
    • View Profile
Re: NRS v1.5.1e
« Reply #66 on: April 15, 2015, 06:27:40 am »

Quote
Why do you think that prunnable parts are "by definition" never parts of the transaction? Prunnable parts are never signed but they can be included in the transaction safely. When they are prunned the sign still will be valid but they will be removed from the transaction.
They are a part of the transaction, but not of the transaction bytes. Only the hash is a part of the bytes.
What is the problem moving them to the bytes? They are not signed so can be easily removed from the bytes.
We cannot add and maintain yet another bytes format, which is not used anywhere in the core. We already support json for the complete transactions, and bytes for the signed part only. To be able to submit the prunable parts to broadcastTransactions, there will be two ways: either use the transactionJSON parameter, which includes everything (and if doing the signing client side, just put the signature in it too), or use the transactionBytes for the signed bytes and put the prunable parts in prunableAttachmentJSON, with the same format as the transaction.attachment JSON (non-prunable parts in it will be ignored) which is returned by all CreateTransaction APIs.

Adding separate message, encryptedMessage, and so on parameters to broadcastTransaction will indeed not scale as we need to add more and more prunable parts, this is why we need to replace them with that single prunableAttachmentJSON parameter in 1.5.2e.
This sounds good. transactionJSON looks like a good solution.
Logged

coinomat

  • Hero Member
  • *****
  • Karma: +214/-18
  • Offline Offline
  • Posts: 1520
    • View Profile
Re: NRS v1.5.1e
« Reply #67 on: April 15, 2015, 11:49:02 am »

This release raises some "philosophical" questions - should blockchain be used for storing arbitrary data at all? Where's the limit for the data volume that could reasonably be stored?  Tough questions IMO. Obviously you can't store a movie on blockchain.
It's hard to say whether this will be an efficient solution, only field tests can show it.
Logged
Time to go further

Tosch110

  • Hero Member
  • *****
  • Karma: +211/-18
  • Offline Offline
  • Posts: 2365
    • View Profile
Re: NRS v1.5.1e
« Reply #68 on: April 15, 2015, 07:02:10 pm »

Would it be possible to add an optional parameter "goods" to the "getDGSPurchases" API?

This could allow to check if a good has be been bought by a user, without scrolling through all the buyer & seller pairs.

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.1e
« Reply #69 on: April 15, 2015, 07:23:06 pm »

Seems more appropriate to add optional "buyer" parameter to getDGSGoodsPurchases?
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.1e
« Reply #70 on: April 15, 2015, 07:28:05 pm »

This release raises some "philosophical" questions - should blockchain be used for storing arbitrary data at all?
It is not storage, it is distribution. But the questions will become even more philosophical in 1.5.3e, in which I hope to include the prunable tagged data feature. I already implemented the UploadTaggedData API and all needed query APIs.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

Tosch110

  • Hero Member
  • *****
  • Karma: +211/-18
  • Offline Offline
  • Posts: 2365
    • View Profile
Re: NRS v1.5.1e
« Reply #71 on: April 15, 2015, 07:48:53 pm »

Seems more appropriate to add optional "buyer" parameter to getDGSGoodsPurchases?

I am satisfied with this aswell :)

EvilDave

  • Hero Member
  • *****
  • Karma: +341/-40
  • Offline Offline
  • Posts: 1789
    • View Profile
    • NXT Foundation
Re: NRS v1.5.1e
« Reply #72 on: April 15, 2015, 11:43:00 pm »

Found it lying around in my garage under some Jaguar parts, mate.  ;D

Anyhow, my block 260281 problem just got wierder: I deleted the entire nxt_test_db directory, and let the blockchain d/l again.
And now I'm stuck on block 260281 again........wtf is going on ? This was on my Ubuntu laptop, btw, still on 1.5.1e.

I uploaded my testnet DB to https://mega.co.nz/#!YAsklRAR!Yaabb1A6d4HYatcw6kxRU_1nfVU8EkIQLwpQqP5xAXc
Unzip file on top of the nxt 1.5.1e root folder, it will overwrite your existing testnet DB.

Much thanks, mate, that got my blockchains back on track........now on to the 1.5.2e thread.  ;D
Logged
Nulli Dei, nulli Reges, solum NXT
NXT Donations: NXT-BNZB-9V8M-XRPW-3S3WD
We will ride eternal, shiny and chrome!
Pages: 1 2 3 [4]  All
 

elective-stereophonic
elective-stereophonic
assembly
assembly