Nxt Forum

Nxt Discussion => Nxt Community News and Announcements => Topic started by: Jean-Luc on April 27, 2014, 01:51:53 pm

Title: Development Roadmap update
Post by: Jean-Luc on April 27, 2014, 01:51:53 pm
To summarize where we are now:

I am now working on integrating into the develop branch the changes from
feature/tf branch that CfB was working on. Those include Digital Goods Store,
Effective Balance Leasing, Transparent Forging changes...

Effective balance leasing is ready and will be enabled on production at block 130,000.

Digital Goods Store is in my view the next critical feature we need to push out, and I will focus my efforts on it. Now working on adding the http API, and when ready will have Wesley start work on the UI. This will all be done in the private repository until production ready. If we work hard on DGS in the next two weeks we would be able to launch it very shortly after AE.

Quote
I also need to integrate the canonical signatures and AES encrypt patches from
Dr Evil, and switch the encryption in DGS to use AES.
The partially-canonical signatures enforcement is done and will become active at NQT_BLOCK in production. The AES encryption (using Bouncy Castle) remains to be implemented as it is needed for the DGS. Either me, or Alex, will work on that.

As part of the NQT_BLOCK transition, I had to implement using full 256-bit hashes for referenced transactions, to counteract collision attacks described by DrEvil. This took several days of work, but had to be done now because it is again a hard fork. And related to referenced transactions, I added a few new API calls, and made escrow transactions easier.

Quote
In the meantime, Wesley and other client devs should make the transition to NQT
so that their clients work with the 0.9 branch.
Wesley has done it, and we will release 1.0.0 next week with his client included. Other clients are expected to follow.

Quote
After bugs in fractional NXT transactions are found and fixed, we need to try on
test net the NQT_BLOCK transition without a blockchain reset.
This was tested, twice actually, so we are ready for the transition in production.

Shortly after the NQT_BLOCK has passed in production, there will be a release in which I remove code that is no longer needed or used - anything related to creating new transactions or blocks in pre-NQT format (which will obviously never need to happen again), removing transaction hashes and replacing them with full hashes instead, relying on the enforcement of canonical signatures instead of hashes for detecting signature malleability attacks, and so on. I have already done most of the work needed, in a private branch.

Quote
I need to work on moving all derived objects - Accounts, Assets, etc, to the
database.
Haven't had the chance to start on this yet. Thinking about priorities, I believe we should finish and push out Digital Goods Store, because this is a unique user-visible feature we have, and is near completion. As soon as I am done with the core support for DGS, while Wesley is working on the UI I will move to work on the database persistence in a separate private branch. I will evaluate using Hibernate for the ORM layer.

Quote
The performance of getNextBlockGenerators, needed for TF, has to be significantly
improved.
I did the performance improvements I could think of, but it only works for accounts that have made hub announcements.

Quote
the implementation of UDP server and accepting transactions over UDP for
faster and DDOS-resistant processing, is again a completely standalone project
that somebody needs to take.
CfB mentioned he would work on that - don't know if he has made any progress?

A few more tasks have come up for other developers:

Add server-side support for Reed-Solomon account numbers.

Add reporting for balance leasing, effective balances, and fees earned, for the needs of balance leasing pool operators. I think the DebugTrace facility could be enhanced to do that, before we have database support.

Title: Re: Development Roadmap update
Post by: Eadeqa on April 27, 2014, 02:07:17 pm
The partially-canonical signatures enforcement is done and will become active at NQT_BLOCK in production. The AES encryption (using Bouncy Castle) remains to be implemented as it is

Some may not like it, but  I would suggest that we just use 128-bits key for AES. It's still just as strong as curve25519 that has 128-bits of security. A lot less hassle.

In any case, what's wrong with my bouncy castle version? (it's exactly same as Dr evil's, uses 256-bit key, with bouncy castle API).

http://bouncycastle.org/download/lcrypto-jdk15on-150.zip

Code: [Select]
public static byte[] aesEncrypt(byte[] plaintext, byte[] myPrivateKey, byte[] theirPublicKey)
    throws IOException, InvalidCipherTextException {
        byte[] dhSharedSecret = new byte[32];
        Curve25519.curve(dhSharedSecret, myPrivateKey, theirPublicKey);
        byte[] key = sha256().digest(dhSharedSecret);
        byte[] iv = new byte[16];
        secureRandom.get().nextBytes(iv);
        PaddedBufferedBlockCipher aes = new PaddedBufferedBlockCipher(new CBCBlockCipher(
            new AESEngine()));
        CipherParameters ivAndKey = new ParametersWithIV(new KeyParameter(key), iv);
        aes.init(true, ivAndKey);
        byte[] output = new byte[aes.getOutputSize(plaintext.length)];
        int len = aes.processBytes(plaintext, 0, plaintext.length, output, 0);
        aes.doFinal(output, len);
        ByteArrayOutputStream ciphertextOut = new ByteArrayOutputStream();
        ciphertextOut.write(iv);
        ciphertextOut.write(output);
        return ciphertextOut.toByteArray();
    }

Code: [Select]
public static byte[] aesDecrypt(byte[] ivCiphertext, byte[] myPrivateKey, byte theirPublicKey[])
    throws InvalidCipherTextException {
        if (ivCiphertext.length < 16 || ivCiphertext.length % 16 != 0) {
            throw new InvalidCipherTextException("invalid ciphertext");
        }
        byte[] iv = Arrays.copyOfRange(ivCiphertext, 0, 16);
        byte[] ciphertext = Arrays.copyOfRange(ivCiphertext, 16, ivCiphertext.length);
        byte[] dhSharedSecret = new byte[32];
        Curve25519.curve(dhSharedSecret, myPrivateKey, theirPublicKey);
        byte[] key = sha256().digest(dhSharedSecret);
        PaddedBufferedBlockCipher aes = new PaddedBufferedBlockCipher(new CBCBlockCipher(
            new AESEngine()));
        CipherParameters ivAndKey = new ParametersWithIV(new KeyParameter(key), iv);
        aes.init(false, ivAndKey);
        byte[] output = new byte[aes.getOutputSize(ciphertext.length)];
        int len = aes.processBytes(ciphertext, 0, ciphertext.length, output, 0);
        aes.doFinal(output, len);
        return output;
    }



Title: Re: Development Roadmap update
Post by: Jean-Luc on April 27, 2014, 02:58:54 pm
I just haven't gotten around to looking at it yet, there is probably nothing wrong. I was going to copy it, then skim through the bouncy castle documentation to make sure I also understand what is going on, and then change the DGS to use that.
Title: Re: Development Roadmap update
Post by: Come-from-Beyond on April 27, 2014, 04:19:14 pm
CfB mentioned he would work on that - don't know if he has made any progress?

I'm waiting for AE and DGS to continue working on MS. UDP is not urgent IMO coz most of the blocks r empty. We simply don't have so many transactions to switch to the next step of TF.
Title: Re: Development Roadmap update
Post by: Daedelus on April 28, 2014, 07:47:16 am
Questions...

Who is Alex?
Is a Hub a hallmarked Node or something different (better)? Another step toward full TF?


(I remember Hub from the megathread as being part of the 'backbone' infrastructure of Nxt)
Title: Re: Development Roadmap update
Post by: Eadeqa on April 30, 2014, 07:03:28 pm
Who is Alex?

Did you ever figure out?
Title: Re: Development Roadmap update
Post by: Daedelus on April 30, 2014, 09:19:57 pm
No. I assumed a hired dev..
 ;D
Title: Re: Development Roadmap update
Post by: bitcoinpaul on April 30, 2014, 09:22:59 pm
A competent java coder working on the core
Title: Re: Development Roadmap update
Post by: Daedelus on April 30, 2014, 09:52:48 pm
Is his other name Come-from-Beyond?  :D :D :D :D :D
Title: Re: Development Roadmap update
Post by: Eadeqa on April 30, 2014, 10:27:30 pm
A competent java coder working on the core

Is he on this forum?
Title: Re: Development Roadmap update
Post by: Eadeqa on May 05, 2014, 05:58:14 pm
The partially-canonical signatures enforcement is done and will become active at NQT_BLOCK in production. The AES encryption (using Bouncy Castle) remains to be implemented as it is

Some may not like it, but  I would suggest that we just use 128-bits key for AES. It's still just as strong as curve25519 that has 128-bits of security. A lot less hassle.

In any case, what's wrong with my bouncy castle version? (it's exactly same as Dr evil's, uses 256-bit key, with bouncy castle API).

http://bouncycastle.org/download/lcrypto-jdk15on-150.zip

Code: [Select]
public static byte[] aesEncrypt(byte[] plaintext, byte[] myPrivateKey, byte[] theirPublicKey)
    throws IOException, InvalidCipherTextException {
        byte[] dhSharedSecret = new byte[32];
        Curve25519.curve(dhSharedSecret, myPrivateKey, theirPublicKey);
        byte[] key = sha256().digest(dhSharedSecret);
        byte[] iv = new byte[16];
        secureRandom.get().nextBytes(iv);
        PaddedBufferedBlockCipher aes = new PaddedBufferedBlockCipher(new CBCBlockCipher(
            new AESEngine()));
        CipherParameters ivAndKey = new ParametersWithIV(new KeyParameter(key), iv);
        aes.init(true, ivAndKey);
        byte[] output = new byte[aes.getOutputSize(plaintext.length)];
        int len = aes.processBytes(plaintext, 0, plaintext.length, output, 0);
        aes.doFinal(output, len);
        ByteArrayOutputStream ciphertextOut = new ByteArrayOutputStream();
        ciphertextOut.write(iv);
        ciphertextOut.write(output);
        return ciphertextOut.toByteArray();
    }

Code: [Select]
public static byte[] aesDecrypt(byte[] ivCiphertext, byte[] myPrivateKey, byte theirPublicKey[])
    throws InvalidCipherTextException {
        if (ivCiphertext.length < 16 || ivCiphertext.length % 16 != 0) {
            throw new InvalidCipherTextException("invalid ciphertext");
        }
        byte[] iv = Arrays.copyOfRange(ivCiphertext, 0, 16);
        byte[] ciphertext = Arrays.copyOfRange(ivCiphertext, 16, ivCiphertext.length);
        byte[] dhSharedSecret = new byte[32];
        Curve25519.curve(dhSharedSecret, myPrivateKey, theirPublicKey);
        byte[] key = sha256().digest(dhSharedSecret);
        PaddedBufferedBlockCipher aes = new PaddedBufferedBlockCipher(new CBCBlockCipher(
            new AESEngine()));
        CipherParameters ivAndKey = new ParametersWithIV(new KeyParameter(key), iv);
        aes.init(false, ivAndKey);
        byte[] output = new byte[aes.getOutputSize(ciphertext.length)];
        int len = aes.processBytes(ciphertext, 0, ciphertext.length, output, 0);
        aes.doFinal(output, len);
        return output;
    }

I think this is not totally correct. Both encryption and decryption should do it like this:

Code: [Select]
      int len2 =  aes.doFinal(output, len);       

      byte[] result = new byte[len+len2];
      System.arraycopy(output, 0, result, 0, result.length);

Title: Re: Development Roadmap update
Post by: Eadeqa on May 05, 2014, 07:27:55 pm
Not sure if it makes any difference for encryption (probably not), but it certainly does for decryption. 

In any case, changing both to this version:

Code: [Select]
public static byte[] aesEncrypt(byte[] plaintext, byte[] myPrivateKey, byte[] theirPublicKey)
    throws InvalidCipherTextException {
        byte[] dhSharedSecret = new byte[32];
        Curve25519.curve(dhSharedSecret, myPrivateKey, theirPublicKey);
        byte[] key = sha256().digest(dhSharedSecret);
        byte[] iv = new byte[16];
        secureRandom.get().nextBytes(iv);
        PaddedBufferedBlockCipher aes = new PaddedBufferedBlockCipher(new CBCBlockCipher(
            new AESEngine()));
        CipherParameters ivAndKey = new ParametersWithIV(new KeyParameter(key), iv);
        aes.init(true, ivAndKey);
        byte[] output = new byte[aes.getOutputSize(plaintext.length)];
        int len = aes.processBytes(plaintext, 0, plaintext.length, output, 0);

       int len2 =  aes.doFinal(output, len);       

       byte[] result = new byte[iv.length+len+len2];
       System.arraycopy(iv, 0, result, 0, iv.length);
       System.arraycopy(output, 0, result, iv.length, result.length-iv.length);
       return result;


    }

Code: [Select]
public static byte[] aesDecrypt(byte[] ivCiphertext, byte[] myPrivateKey, byte theirPublicKey[])
    throws InvalidCipherTextException {
        if (ivCiphertext.length < 16 || ivCiphertext.length % 16 != 0) {
            throw new InvalidCipherTextException("invalid ciphertext");
        }
        byte[] iv = Arrays.copyOfRange(ivCiphertext, 0, 16);
        byte[] ciphertext = Arrays.copyOfRange(ivCiphertext, 16, ivCiphertext.length);
        byte[] dhSharedSecret = new byte[32];
        Curve25519.curve(dhSharedSecret, myPrivateKey, theirPublicKey);
        byte[] key = sha256().digest(dhSharedSecret);
        PaddedBufferedBlockCipher aes = new PaddedBufferedBlockCipher(new CBCBlockCipher(
            new AESEngine()));
        CipherParameters ivAndKey = new ParametersWithIV(new KeyParameter(key), iv);
        aes.init(false, ivAndKey);
        byte[] output = new byte[aes.getOutputSize(ciphertext.length)];
        int len = aes.processBytes(ciphertext, 0, ciphertext.length, output, 0);
     
        int len2 =  aes.doFinal(output, len);       
       
        byte[] result = new byte[len+len2];
        System.arraycopy(output, 0, result, 0, result.length);
        return result;       
    }
Title: Re: Development Roadmap update
Post by: Jean-Luc on May 06, 2014, 12:24:53 am
I am done with the implementation of the Digital Goods Store API, and have switched to BouncyCastle for the AES encryption. Now it is assigned to Wesley to start working on the UI.
Title: Re: Development Roadmap update
Post by: CoinTropololis_JustaBit on May 06, 2014, 12:25:46 am
I am done with the implementation of the Digital Goods Store API, and have switched to BouncyCastle for the AES encryption. Now it is assigned to Wesley to start working on the UI.

Way to go Jean-Luc, you are a beast!
Title: Re: Development Roadmap update
Post by: coolfish on May 06, 2014, 04:19:13 pm
I am done with the implementation of the Digital Goods Store API, and have switched to BouncyCastle for the AES encryption. Now it is assigned to Wesley to start working on the UI.

wow great news!
Title: Re: Development Roadmap update
Post by: msin on May 06, 2014, 04:57:07 pm
I am done with the implementation of the Digital Goods Store API, and have switched to BouncyCastle for the AES encryption. Now it is assigned to Wesley to start working on the UI.

Great work JL, thank you, hopefully we'll get an update from Welsey on UI implementation timeframe.
Title: Re: Development Roadmap update
Post by: Daedelus on May 15, 2014, 11:26:07 am
Who is Alex?

Did you ever figure out?

Kushti, I believe  ;D

Congrats on the AE implementation  ;D It seems to be attracting real businesses, new and existing, and creating a real buzz. jl777 especially is spamming it every day with fantastical projects he is churning out  ;D MSC and XCP's exchange never appeared in my radar before they sank back down so this is very positive to me.

Any chance of another 1 month plan, now NQT, AE and Leasing have been implemented seamlessly?  (At least from the outside looking in)  With Wesley too now I see, none of you guys must sleep. I guess Alias Transfer will be next with the DGS?

Thanks  ;D
Title: Re: Development Roadmap update
Post by: jl777 on May 15, 2014, 12:19:01 pm
Who is Alex?

Did you ever figure out?

Kushti, I believe  ;D

Congrats on the AE implementation  ;D It seems to be attracting real businesses, new and existing, and creating a real buzz. jl777 especially is spamming it every day with fantastical projects he is churning out  ;D MSC and XCP's exchange never appeared in my radar before they sank back down so this is very positive to me.

Any chance of another 1 month plan, now NQT, AE and Leasing have been implemented seamlessly?  (At least from the outside looking in)  With Wesley too now I see, none of you guys must sleep. I guess Alias Transfer will be next with the DGS?

Thanks  ;D
I sure could use community help to finalize multigateway and get NXTservices tested, code reviewed, etc.
I will go back into dev mode now that my financing phase is winding down for the month. However, like the werewolf I will morph back into finance mode once a month. When is the next full moon?

James
Title: Re: Development Roadmap update
Post by: Daedelus on May 15, 2014, 12:29:14 pm
Who is Alex?

Did you ever figure out?

Kushti, I believe  ;D

Congrats on the AE implementation  ;D It seems to be attracting real businesses, new and existing, and creating a real buzz. jl777 especially is spamming it every day with fantastical projects he is churning out  ;D MSC and XCP's exchange never appeared in my radar before they sank back down so this is very positive to me.

Any chance of another 1 month plan, now NQT, AE and Leasing have been implemented seamlessly?  (At least from the outside looking in)  With Wesley too now I see, none of you guys must sleep. I guess Alias Transfer will be next with the DGS?

Thanks  ;D
I sure could use community help to finalize multigateway and get NXTservices tested, code reviewed, etc.
I will go back into dev mode now that my financing phase is winding down for the month. However, like the werewolf I will morph back into finance mode once a month. When is the next full moon?

James

Is NXTServices ready for total noobs to try and play with it? I am always primed to spam the pub threads...
Title: Re: Development Roadmap update
Post by: bitcoinpaul on May 15, 2014, 12:31:11 pm
Who developed nxtservices?
Title: Re: Development Roadmap update
Post by: Daedelus on May 15, 2014, 12:37:11 pm
Who developed nxtservices?

jl777. It has his Nxtsubatomic and Nxtatomic in there with encypted chat (and probably other cool bits and pieces)
Title: Re: Development Roadmap update
Post by: bitcoinpaul on May 15, 2014, 12:46:00 pm
Wait, nxtservices code will be in the core - but james tinkered on top of the core, right?

confused services with service providers, I guess.
Title: Re: Development Roadmap update
Post by: jl777 on May 15, 2014, 12:49:27 pm
Wait, nxtservices code will be in the core - but james tinkered on top of the core, right?

confused services with service providers, I guess.
My codebase is all C, no way it will be in the core
Title: Re: Development Roadmap update
Post by: bitcoinpaul on May 15, 2014, 12:50:16 pm
Who's on service providers?
Title: Re: Development Roadmap update
Post by: jl777 on May 15, 2014, 12:50:23 pm
Who is Alex?

Did you ever figure out?

Kushti, I believe  ;D

Congrats on the AE implementation  ;D It seems to be attracting real businesses, new and existing, and creating a real buzz. jl777 especially is spamming it every day with fantastical projects he is churning out  ;D MSC and XCP's exchange never appeared in my radar before they sank back down so this is very positive to me.

Any chance of another 1 month plan, now NQT, AE and Leasing have been implemented seamlessly?  (At least from the outside looking in)  With Wesley too now I see, none of you guys must sleep. I guess Alias Transfer will be next with the DGS?

Thanks  ;D
I sure could use community help to finalize multigateway and get NXTservices tested, code reviewed, etc.
I will go back into dev mode now that my financing phase is winding down for the month. However, like the werewolf I will morph back into finance mode once a month. When is the next full moon?

James

Is NXTServices ready for total noobs to try and play with it? I am always primed to spam the pub threads...
Not even close. No GUI at all. as easy as using unix command line :)
Title: Re: Development Roadmap update
Post by: Daedelus on May 15, 2014, 12:54:47 pm
Who is Alex?

Did you ever figure out?

Kushti, I believe  ;D

Congrats on the AE implementation  ;D It seems to be attracting real businesses, new and existing, and creating a real buzz. jl777 especially is spamming it every day with fantastical projects he is churning out  ;D MSC and XCP's exchange never appeared in my radar before they sank back down so this is very positive to me.

Any chance of another 1 month plan, now NQT, AE and Leasing have been implemented seamlessly?  (At least from the outside looking in)  With Wesley too now I see, none of you guys must sleep. I guess Alias Transfer will be next with the DGS?

Thanks  ;D
I sure could use community help to finalize multigateway and get NXTservices tested, code reviewed, etc.
I will go back into dev mode now that my financing phase is winding down for the month. However, like the werewolf I will morph back into finance mode once a month. When is the next full moon?

James

Is NXTServices ready for total noobs to try and play with it? I am always primed to spam the pub threads...
Not even close. No GUI at all. as easy as using unix command line :)

Can you only test multigateway through NXTservices? I think I am confused again  ::)

 ;D
Title: Re: Development Roadmap update
Post by: jl777 on May 15, 2014, 12:57:33 pm
Who is Alex?

Did you ever figure out?

Kushti, I believe  ;D

Congrats on the AE implementation  ;D It seems to be attracting real businesses, new and existing, and creating a real buzz. jl777 especially is spamming it every day with fantastical projects he is churning out  ;D MSC and XCP's exchange never appeared in my radar before they sank back down so this is very positive to me.

Any chance of another 1 month plan, now NQT, AE and Leasing have been implemented seamlessly?  (At least from the outside looking in)  With Wesley too now I see, none of you guys must sleep. I guess Alias Transfer will be next with the DGS?

Thanks  ;D
I sure could use community help to finalize multigateway and get NXTservices tested, code reviewed, etc.
I will go back into dev mode now that my financing phase is winding down for the month. However, like the werewolf I will morph back into finance mode once a month. When is the next full moon?

James

Is NXTServices ready for total noobs to try and play with it? I am always primed to spam the pub threads...
Not even close. No GUI at all. as easy as using unix command line :)

Can you only test multigateway through NXTservices? I think I am confused again  ::)

 ;D
Yes, that would be the easiest. You need to put in your wallet addresses and copy and paste tokenized URL
It really needs to be put into a GUI. maybe what we need the most is a GUI dev?
yes, OK I will take three GUI devs please
elective-stereophonic
elective-stereophonic
assembly
assembly