elective-stereophonic
elective-stereophonic
verifyPrunableMessage in Javascript or in plain english singapore
Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Stable Nxt Client: Nxt 1.11.15 | Latest Experimental Nxt Client: Nxt 1.12.0e

Author Topic: verifyPrunableMessage in Javascript or in plain english  (Read 2105 times)

websioux

  • Sr. Member
  • ****
  • Karma: +69/-1
  • Offline Offline
  • Posts: 343
  • Great changes grow bottom up
    • View Profile
    • Scriba.io the Blockchain Scribe
verifyPrunableMessage in Javascript or in plain english
« on: January 25, 2016, 06:50:39 pm »

Here is a prunable message transaction :

Code: [Select]
{
    "senderPublicKey": "420468dfe267571e27429920d215de6bcbcc930dde1ab485dbc4f1b51197fb01",
    "signature": "65faced1a7b76cba99e6d07df0a77545284d16f2ae4d0e2cf9d7b378015c740a28a3dd719aac36d50141b805ea7f7a14cd5d3c5a666e878ebb483ba7b89a9539",
    "feeNQT": "100000000",
    "transactionIndex": 1,
    "requestProcessingTime": 0,
    "type": 1,
    "confirmations": 20,
    "fullHash": "f51983f24ad53ff554c33c5409fdf4c07712f4fae74b0b0689a57f8339eb8bbe",
    "version": 1,
    "phased": false,
    "ecBlockId": "9167853488908390031",
    "signatureHash": "bcc4ac2e7b70cb3bc3b165e751f2890e8f5e1307f7e8d60ce162f175674a86af",
    "attachment": {
        "version.PrunablePlainMessage": 1,
        "messageIsText": true,
        "messageHash": "d48e7cbc08b6ff1fc755b49d90d56d0fc347cd4e2b3ba7c4eaade6ac36a94d3e",
        "message": "db8b6b796c56927a41d9f195407ae712b71514343c420fdf75c95b5cfa69c47a",
        "version.ArbitraryMessage": 0
    },
    "senderRS": "NXT-39LJ-V67N-TJB9-EHUKA",
    "subtype": 0,
    "amountNQT": "0",
    "sender": "14143360948395613776",
    "ecBlockHeight": 624346,
    "block": "8168468208213467054",
    "blockTimestamp": 68426470,
    "deadline": 1440,
    "transaction": "17672077980698089973",
    "timestamp": 68426434,
    "height": 624355
}

I know that at anytime the API allows to verify the message that will be pruned thanks to this call :

http://nxt.scriba.io:7876/nxt?requestType=verifyPrunableMessage&transaction=17672077980698089973&message=db8b6b796c56927a41d9f195407ae712b71514343c420fdf75c95b5cfa69c47a

Which returns ok

Code: [Select]
{
    "version.PrunablePlainMessage": 1,
    "verify": true,
    "messageIsText": true,
    "messageHash": "d48e7cbc08b6ff1fc755b49d90d56d0fc347cd4e2b3ba7c4eaade6ac36a94d3e",
    "requestProcessingTime": 1,
    "message": "db8b6b796c56927a41d9f195407ae712b71514343c420fdf75c95b5cfa69c47a"
}

But I would like to verify it in javascript and I keep failing. My real goal is to explain it such that someone else can do it too so I need to succeed it first.

In the java source, I found in Appendix.PrunablePlainMessage

Code: [Select]
            MessageDigest digest = Crypto.sha256();
            digest.update((byte)(isText ? 1 : 0));
            digest.update(message);
            return digest.digest();


which seems to be how "message" is transformed to "messagehash"
The wiki seems to confirm but is not accurate enough : http://wiki.nxtcrypto.org/wiki/The_Nxt_API#Verify_Prunable_Message

I have tried to concat 1+message, '1'+message, true+message, converters.int32ToBytes(1)+message, converters.int32ToBytes(true)+message and all kind of combinations I could think of but with no success.

I've used different libraries, which all gives the same, so my problem is really that I'm missing something.

Does anyone knows what I can do to go from "message" to "messagehash" ?

I believe this is a general useful question since we should not only rely on the API to verify that some prunedmessage retreived is correct.
Logged
Secret Miner <= communicate with style | NotBot <= timestamp digital docs

Riker

  • Core Dev
  • Hero Member
  • *****
  • Karma: +439/-42
  • Offline Offline
  • Posts: 1796
    • View Profile
Re: verifyPrunableMessage in Javascript or in plain english
« Reply #1 on: January 26, 2016, 08:33:48 am »

Take a look at nrs.server.js line 1424 and on. You can see how the client validates the server transaction bytes for PrunablePlainMessage.
The code responsible for reproducing the server hash calculation starts at line 1432:
sha256 = CryptoJS.algo.SHA256.create();

First we use sha256.update() to add the isText flag as byte 0 or 1 and then message after the necessary conversions. Then we call sha256.finalize() to calculate the hash and compare it to the hash produced by the server.
Note that the conversions from word array to byte array and vice versa are required by the CryptoJS library we are using. If you are using a different library to calculate the sha256 hash, you will probably need to serialize the data differently.
Logged
NXT Core Dev
Account: NXT-HBFW-X8TE-WXPW-DZFAG
Public Key: D8311651 Key fingerprint: 0560 443B 035C EE08 0EC0  D2DD 275E 94A7 D831 1651

websioux

  • Sr. Member
  • ****
  • Karma: +69/-1
  • Offline Offline
  • Posts: 343
  • Great changes grow bottom up
    • View Profile
    • Scriba.io the Blockchain Scribe
Re: verifyPrunableMessage in Javascript or in plain english
« Reply #2 on: January 26, 2016, 11:02:38 am »

Hello Riker, thank you, your reply is sharp, correct and fast !

Thanks to it, I managed to reproduce with with CryptoJS

Code: [Select]

var isText = [1];
var message = 'db8b6b796c56927a41d9f195407ae712b71514343c420fdf75c95b5cfa69c47a';

console.log('isText : ',isText);
console.log('message : ',message);

sha256 = CryptoJS.algo.SHA256.create();
sha256.update(converters.byteArrayToWordArrayEx(isText));
sha256.update(converters.byteArrayToWordArrayEx(getUtf8Bytes(message)));
hashWords = sha256.finalize();
calculatedHash = converters.wordArrayToByteArrayEx(hashWords);
outputHash = converters.byteArrayToHexString(calculatedHash)

console.log('messageHash : ',outputHash);

If someone else is reading : getUtf8Bytes is taken from nrs.util.js and the converters functions from nrs.converters.js

which nicely outputs :

isText :  [1]
message :  db8b6b796c56927a41d9f195407ae712b71514343c420fdf75c95b5cfa69c47a
messageHash :  d48e7cbc08b6ff1fc755b49d90d56d0fc347cd4e2b3ba7c4eaade6ac36a94d3e

Now I understand that I have to look around these word array conversions to reproduce with others, but I should be able to.. will see..
Logged
Secret Miner <= communicate with style | NotBot <= timestamp digital docs

websioux

  • Sr. Member
  • ****
  • Karma: +69/-1
  • Offline Offline
  • Posts: 343
  • Great changes grow bottom up
    • View Profile
    • Scriba.io the Blockchain Scribe
Re: verifyPrunableMessage in Javascript or in plain english
« Reply #3 on: January 26, 2016, 04:43:20 pm »

In fact it doesn't seems so easy to reproduce.

It reduces to :

Code: [Select]
var isText = [1];
var message = 'db8b6b796c56927a41d9f195407ae712b71514343c420fdf75c95b5cfa69c47a';
var wa1 = byteArrayToWordArrayEx(isText);
sha256 = CryptoJS.algo.SHA256.create();
sha256.update(wa1);
sha256.update(message);
hashWords = sha256.finalize();
console.log('messageHash : ',hashWords.toString(CryptoJS.enc.Hex));

function byteArrayToWordArrayEx: function (u8arr) {
            var len = u8arr.length;
            var words = [];
            for (var i = 0; i < len; i++) {
                words[i >>> 2] |= (u8arr[i] & 0xff) << (24 - (i % 4) * 8);
            }
            return CryptoJS.lib.WordArray.create(words, len);
        }


But then, I'm wondering why it was chosen to add the flag (isText) in this specific wordArray format, which creates a cryptoJs strong dependancy instead of a simple concatenation of the message with a "1" or a "0".
What prevented a simple concat ?
CrypoJs accepts both string and wordArray format, it converts to wordArray if it's a string to make its hash calc.
So, in my code, wa1 could have been "1" and then verification would be straight forward whatever native library.

Instead of one line of explanation I will have to write a full text that nobody will ever be willing to read and this will ruin the confidence they will have in the service.

If you don't know, it doesn't mind, I understand, but if you do, then I'm interested in learning something that may be fundamental or not, may be coding went this way.. or I may be I'm adding complexity where there is not ?

My whole idea behind all of this was that instead of paying 2 NXT to write a sha256 hash with permanent messages, I hoped I could provide my user the formula that gives the message_hash (which is permanent) and then use prunablemessages at 1NXT with my user hash..
But if I do, I have to say to use cryptojs lib or verify with nxt api instead of saying just "hash your hash with this or this modification" and you will find the blockchain record.

They could verify with many independant online tools which gives a feeling of openness and truth, now if I understand correct, they can not. They must have a node runing, or trust one to verify for them, or make a too much complicated use of a library.

So it looks like I have to use permanent messages instead, which is not nice since I'm using the same blockchain size than prunablemessages (even less since I'm using the same size as an empty prunable message) but I'm paying twice..


Logged
Secret Miner <= communicate with style | NotBot <= timestamp digital docs

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: verifyPrunableMessage in Javascript or in plain english
« Reply #4 on: January 26, 2016, 10:56:33 pm »

I don't know why it is complicated to do in javascript the equivalent of how the sha256 digest is calculated in java, as you have seen, the java code is very simple.

But in your case, there is no need to use prunable messages. If all you want to store as a message is a 32-byte sha256 hash, it will fit exactly in the free size allowed for a permanent message. You need to specify messageIsText=false though, so that instead of storing the string db8b6b796c56927a41d9f195407ae712b71514343c420fdf75c95b5cfa69c47a as 64 characters, which would take 64 bytes, it is interpreted as the hex-encoded binary it actually is, and fits in 32 bytes.

Also, a message does not need to have a recipient, instead of using the same account as sender and recipient you can leave the recipient field empty.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

websioux

  • Sr. Member
  • ****
  • Karma: +69/-1
  • Offline Offline
  • Posts: 343
  • Great changes grow bottom up
    • View Profile
    • Scriba.io the Blockchain Scribe
Re: verifyPrunableMessage in Javascript or in plain english
« Reply #5 on: January 27, 2016, 02:31:01 pm »

Thank you Jean Luc, you solved my problem in one line of text  :)

Non prunable with messageIsText=false will be perfect for me and Nxt blockchain size to store my sha256 in 32-bytes only instead of 64 when it is considered as text, I get it now.
I will also not use recipient anymore to show good practice in this situation.

Regarding this java line : digest.update((byte)(isText ? 1 : 0)) involved in verifyPrunable :

I do not know Java but I guess that if it would have been like this : digest.update((isText ? "1": "0"));
Then, their would be direct easy compatibility with JS and PHP and probably others. Perhaps their still is an easy way, but I couldn't yet find it, and I will stop searching now.

CryptoJS is an open library which can be used the way Java does, but others don't. I'm using forge.js since for large file it appeared to be faster. It also works with progressive hashing but it doesn't use the wordArray format, or a different one (like Riker suggested). I also looked around php hash_update which looked similar but couldn't find how to input this single bit flag in it such it produces the same result.

Anyway, this is how programming is tricky, one misplaced bit and the whole business may fall apart. :P

Hopefully you anticipated this "basic" case with isText is false. There is so much that can be done with this hash anchoring.
Logged
Secret Miner <= communicate with style | NotBot <= timestamp digital docs

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: verifyPrunableMessage in Javascript or in plain english
« Reply #6 on: January 27, 2016, 04:31:50 pm »

The MessageDigest.update() and digest() methods in Java only operate on bytes, something like digest.update("1") will not work. Using the characters '1' and '0' and casting them to bytes would have worked, but would give different values than just 1 and 0 as bytes. Anyway, it is calculated the way it is, we can't change that without breaking all already created message hashes.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

websioux

  • Sr. Member
  • ****
  • Karma: +69/-1
  • Offline Offline
  • Posts: 343
  • Great changes grow bottom up
    • View Profile
    • Scriba.io the Blockchain Scribe
Re: verifyPrunableMessage in Javascript or in plain english
« Reply #7 on: January 27, 2016, 06:09:00 pm »

I understand. Perhaps in case where prunable messages will be usefull, the way it is, is good enough. My case was different, solved differently, so the right way.
Logged
Secret Miner <= communicate with style | NotBot <= timestamp digital docs
 

elective-stereophonic
elective-stereophonic
assembly
assembly