elective-stereophonic
elective-stereophonic
Ramblings and Updates
Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Nxt Client: Nxt 1.11.15

Author Topic: Ramblings and Updates  (Read 3209 times)

jones

  • Board Moderator
  • Hero Member
  • ****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Ramblings and Updates
« on: November 16, 2014, 04:59:31 am »

To start off: some brainstorming logs.

jonesnxt [8:59 PM]
my current plan is more of a "noweight" wallet, as there is very little to no storage of any type on the actual jay node

jonesnxt [9:00 PM]
only a bit of caching for higher speeds

----- Today November 15th, 2014 -----

jonesnxt [7:59 PM]
alright

jonesnxt [8:00 PM]
I've been doing some serious thinking and think I came up with a good way to avoid spoofing, and lower the risk of sybil

jonesnxt [8:00 PM]
some more thought needs to be put into sybil though

jonesnxt [8:02 PM]8:02
So basically the client has a verification process called "verification hops" the client will be required to send the user to a completely different node on a special page that checks the node the user came from and then forwards to two other nodes that also verify the server that the user wants to access before they are sent back to the original page and are checked off and happy that this node is not tampered with

jonesnxt [8:04 PM]
There can still be four servers that collude with each other and continually spoof the verification to the home server, but my thought is that these servers will have all use each other to verify their own server, or they will be caught by other real servers

jonesnxt [8:06 PM]
There can be "node explorers" that poll all of the nodes to discover these verification rings and prove that they are colluding and not safe to use. These explorers can also find the healthiest paths that nodes take and rank nodes based on how spread out all of their verification servers are and how many nodes link them into their verification loops

jonesnxt [8:07 PM]
I also thought that Jay nodes can be used for specialized services by providing a default layout option in the config file

jonesnxt [8:09 PM]
this way a site like secureae could setup a secureae plugin through Jay and put it with a few other useful plugins as a default and run a Jay node, so the secureae is verified, and the user doesn't have to think about customizing and adding plugins
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

jones

  • Board Moderator
  • Hero Member
  • ****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Re: Ramblings and Updates
« Reply #1 on: November 26, 2014, 08:05:33 pm »

Added some more inf. to the code, and did some more rambling

jonesnxt [9:35 PM]
Decided to have a code all-nighter because I have nothing better to do :simple_smile: I'll see how far I can get with Jay in the next seven hours or so

jl777 [9:36 PM]
:simple_smile:

jonesnxt [11:51 PM]
Put some more work into the distributed consensus

jonesnxt [11:52 PM]
first, all nodes on the nodelist (nxt) are public nodes with cors enabled and hallmarked (around 20 good ones)

jonesnxt [11:53 PM]
each api request is sent to three randomly selected servers in parallel.

jonesnxt [11:53 PM]
if the three servers are in complete agreement on the data, then it is deemed usable

jonesnxt [11:54 PM]
if there is any discrepancy, another round is called, and two more randomly selected servers go into the mix

jonesnxt [11:55 PM]
with the now 5 servers polled, if the second round are in agreement and side with any of the first rounds data, that data is selected.

jonesnxt [11:57 PM]
Otherwise it starts breaking down the data and finds consensus if it can, I might add a blacklist if there are servers that continuously out voted by the other nodes

jonesnxt [11:59 PM]
So basically at any given request, the best case scenario being n where n is equal to the node with the slowest response, the less likely edge case gives us 2n and the very rare, basically under attack case gives us 3n

----- Today November 26th, 2014 -----

jonesnxt [12:00 AM]
just some ramblings :simple_smile: I've found this is a good place to get my ides clear, so there might be a lot on here by the end of the night

jonesnxt [12:04 AM]
and in tests I optimistically place n as around 300 ms, with a worst case scenario of 1000 ms

jonesnxt [2:30 AM]
Dealing with cross domain in js is the worst, I think I finally found a semi-acceptable method, at that, calling it a day, I still got quite a bit done, inf. wise at least

btcddev [5:49 AM]
Sounds good. Yes, cors is rough

jl777 [10:10 AM]
I have noticed the DB version of NXT is much, much slower than the RAM based, so 1 second times will be bottlenecked by the speed of NXT DB

jl777 [10:10 AM]
in other words 1 second time is just fine
 
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

blackyblack1

  • Hero Member
  • *****
  • Karma: +165/-82
  • Offline Offline
  • Posts: 1763
    • View Profile
Re: Ramblings and Updates
« Reply #2 on: November 27, 2014, 07:27:09 am »

Maybe I will setup 100 nodes to spoof the network. How do you like the idea?
Logged

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: Ramblings and Updates
« Reply #3 on: November 27, 2014, 07:42:33 am »

Maybe I will setup 100 nodes to spoof the network. How do you like the idea?
OK, this would be a good test when we are ready.
What do you think about having a few whitelisted nodes that are included as a reference?
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

blackyblack1

  • Hero Member
  • *****
  • Karma: +165/-82
  • Offline Offline
  • Posts: 1763
    • View Profile
Re: Ramblings and Updates
« Reply #4 on: November 27, 2014, 07:48:27 am »

Maybe I will setup 100 nodes to spoof the network. How do you like the idea?
OK, this would be a good test when we are ready.
What do you think about having a few whitelisted nodes that are included as a reference?
I'd prefer having a fully centralized solution than making a pseudo-decentralized system. I guess the decentralized consesus only achieved with pow or pos. Don't really know so if you are sure that it won't be broken - go ahead.
Logged

jones

  • Board Moderator
  • Hero Member
  • ****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Re: Ramblings and Updates
« Reply #5 on: November 27, 2014, 08:14:22 am »

Maybe I will setup 100 nodes to spoof the network. How do you like the idea?
OK, this would be a good test when we are ready.
What do you think about having a few whitelisted nodes that are included as a reference?
I'd prefer having a fully centralized solution than making a pseudo-decentralized system. I guess the decentralized consesus only achieved with pow or pos. Don't really know so if you are sure that it won't be broken - go ahead.

it is true that you need some sort of pow or pos system to maintain decentralized consensus is a block chain like system, but jay is only using the network to be sure that the API calls it receives are the right ones. It doesn't have to worry about double spends or the byzantine generals problem because nxt already has that covered, the issue is what you brought up with the Sybil attack which is why for now the node list is composed of 18 nodes that have been hallmarked, all have different ips, and all have cors enabled. There is a chance that there could be three or more colluding to send the same incorrect response to a request, but most of the time the requests won't even be life changing, all the work of making a bunch of servers all to collude for an incorrect getBalance before one of the servers is paired with an honest node and does checks until it blacklists the dishonest one.

its always good to have constructive criticism, keeps me on my toes :)
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

jones

  • Board Moderator
  • Hero Member
  • ****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Re: Ramblings and Updates
« Reply #6 on: February 27, 2015, 06:17:00 am »

Its about time for a Jay update, there's been a lot of shifting around in the thought of this project (as there has always been) :)

to recap, this started as a simple light wallet implementation, but I ended up in a thought problem on how to make a easy online plugin based system for cryptocurrencies, and I've been looking toward a good solution.

A while ago I was analyzing all the things I want Jay to do and I found they fall into 4 main categories:
1. Request - get data from public nodes until I find it trustworthy, so I don't need a public node to use nxt
2. Tx - easily generate transactions from the js so that any code can easily add wallet features
3. Verify - somehow validate the code we downloaded so there aren't any security issues
4. Plugin - Allow people to easily create systems that can be dynamically loaded into a wallet.

These steps get harder and harder as they go on because they all rely on previous ones, plus my ideas on how Verify works are still spotty. (its possible, but would be a large amount of overhead)

A couple weeks ago I had the idea of Vapor while thinking about transaction bytes and downloadable html.

Vapor allows you to sign and broadcast transactions on a single html page without any network present, so security isn't an issue.

Then after that I came up with another cool realization, If I make a smaller program that can review, then sign and broadcast transaction bytes that I give it, then all I need to do is make sure that this tiny program is secure and then I can make extremely unsecure wallets on the internet and not have to worry because my passphrase is never involved.

So this is what I've been experimenting with, a smaller program I like to call Run. I am working on a shortened Tx byte order for it to make the strings shorter (we don't need signature, sender pub key or reftx), so the unsigned bytes get fairly small, I've also been using a base62 encoding on them to make the string shorter, but still copy-pasteable. I coll this TRF (transaction request format) and it looks like this: "TX_wdlUXGP1yHtwrcyFiKgAPCUHKoxfmgrOKUE4jXpxd"

With this I can modify the earlier goals of Jay, as Run now handles the security:
1: Request (stays the same)
2: Tx - Now generates the TRF format unsigned transactions for Run to deal with as the user wants.
3: Verify - No need to now, if someone gives us a malicious TRF string, we can easily see it when Run tells us the tx overview before we accept it
4: Plugin - Now no centralized system of nodes is needed, just allow people to make requests and transactions as they want and let them put them wherever, services that host plugins for this can be made also.

So I think this would be a big step in the right direction.

For the average user the system would work like this:
1. Download and run the small packaged software, Run, and it automatically sets up an nxt account for me and tells me its address. (Having this as a browser plugin would be really cool too)
2. Now open any unsecure nxt plugin website that uses the Jay framework.
3. Browse through the site, lets say its crowdfunding, and I say I want to crowdfund a project, so I click create transaction request
4. the page gives me a string TX_... that I now copy onto my clipboard.
5. now I go back to my copy of Run on my desktop and paste the TRF string then press go
6. Run brings up the tx overview and wants me to make sure everything is correct, it is so I say ok.
7. The transaction is signed by Run then broadcasted to the network, all is good.

So that's the latest on this system, I am working on Run right now (if I can ever pick the right Java swing layout) and will start making Jay work together with it soon enough. :)
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

Riker

  • Core Dev
  • Hero Member
  • *****
  • Karma: +439/-42
  • Offline Offline
  • Posts: 1794
    • View Profile
Re: Ramblings and Updates
« Reply #7 on: February 27, 2015, 06:58:19 am »

Thanks for the update, sounds like you are on to something really useful.

Questions:
Is it really important to use your own proprietary TRF format for this ?
It sounds to me like a premature optimization since this would make the life of plugin developers more difficult at the benefit of slightly reducing network traffic which can be compressed anyway.
Why not just support the hex transaction bytes format provided by the NRS ?
Wouldn't this help plugin developers support Jay without any dependency on Jay itself or easily port existing plugins to work with Jay ?

Regarding the RUN client code, did you consider running it a Signed Java Applet from your website so that users does not have to install it on their system ? You can obtain open source code signing certificate somewhere, perhaps even for free.
This would help reducing trust issues around your RUN code.
And please find a better name than "RUN" maybe TransactionGenerator ?
Logged
NXT Core Dev
Account: NXT-HBFW-X8TE-WXPW-DZFAG
Public Key: D8311651 Key fingerprint: 0560 443B 035C EE08 0EC0  D2DD 275E 94A7 D831 1651

rudeboi

  • Hero Member
  • *****
  • Karma: +55/-4
  • Offline Offline
  • Posts: 633
  • Nxt Organization Member
    • View Profile
Re: Ramblings and Updates
« Reply #8 on: February 27, 2015, 12:45:38 pm »

Thanks for the update Jones, looks like you have some great plans.

Could you clarify point 7:
7. The transaction is signed by Run then broadcasted to the network, all is good.
How are you broadcasting to the network?

Another use for Run:

Let’s say that Run is a browser plugin. With current read only changes to NRS UI, more people start using trusted online public nodes as read only Nxt wallets (blockchain explorers). The existence of Run could enable people to use online nodes with full access.  If they wanted to send a transaction instead of entering passphrase into the UI (which would be prevented on public nodes), they click a button in the UI similar to ‘send to SABnzb’ that pushes the request to Run. Run then shows up a summary of the transaction to confirm, then user signs with the passphrase, once signed Run presents the user with the hex transaction bytes/RTF code and they copy and paste back into the online public node, and the transaction is broadcast to the network.

This way users can use the fully fledged official NRS UI securely anywhere in the world with only a lightweight browser plugin.   
Logged
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬  ▄▀▀▀▀▀▀▀▀▄  ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬●  nimirum  ●▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
▬▬▬ ◖ENDING CENSORSHIP ONLINE◗  ◖ ICO OPEN NOW◗ ▬▬▬

jones

  • Board Moderator
  • Hero Member
  • ****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Re: Ramblings and Updates
« Reply #9 on: February 28, 2015, 02:04:03 am »

Thanks for the update, sounds like you are on to something really useful.

Questions:
1. Is it really important to use your own proprietary TRF format for this ?
2. It sounds to me like a premature optimization since this would make the life of plugin developers more difficult at the benefit of slightly reducing network traffic which can be compressed anyway.
3. Why not just support the hex transaction bytes format provided by the NRS ?
4. Wouldn't this help plugin developers support Jay without any dependency on Jay itself or easily port existing plugins to work with Jay ?

5. Regarding the RUN client code, did you consider running it a Signed Java Applet from your website so that users does not have to install it on their system ? You can obtain open source code signing certificate somewhere, perhaps even for free.
6. This would help reducing trust issues around your RUN code.
7. And please find a better name than "RUN" maybe TransactionGenerator ?

Thanks for the feedback :)

1. I really like my format in this case.
2. The benefit is not so much for network traffic as for ease of use for the user
3. I can make Run work with unsigned tx bytes as provided by NRS as well, I just found TRF to have some cool properties.
4. The structure is a bit different as the normal plugin would just sign and broadcast the tx itself, here we have a step in the middle where we need unsigned tx bytes. Once again, we can use the unsigned tx bytes with Run just fine, I just like TRF for ease of use.

To outline my thoughts of TRF:(basically fancy unsigned transaction bytes)
- The purpose is to get raw unsigned transaction data from the client to the client so it knows what to do.
- Many of the bytes in the normal unsigned transaction bytes are just blank space, which isn't needed for this purpose.
- TRF takes the same basic order of normal transactions but removes the sender public key (as we will put this in ourselves in Run), the reftx (you can include it if you want, but it isn't generally used), and the signature (just zeros).
- I also convert to base62 and add TX_ to make it easily seeable and copy pasteable, also small enough to fit in something like a tweet.
- Becuase the TRF string is not person dependant (sender key is blank) and mostly not time dependant (you can leave out time if you want) you can post TRF strings anywhere in plain text as a quick donate feature. I make a blog post with the string TX_XXXX... (Donate 100 nxt) at the end of my post and people can do so by simply copy pasting it.
- Also it has a specific format, the "TX_", so it can be easily picked up by a regex so a browser plugin of Run can be made to create a popup window over each TRF string describing the tx that would be made if the string was used.

5. This seems cool, I will look into this.
6. yes it would, I will try to keep Run as simple as possible to make it easy to verify. Code signing would help.
7. Personally I like Run becuase it is short, TransactionGenerator is a bit too wordy for me but if anyone can come up with something short and good I will use it, run is a working name at the moment.
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

jones

  • Board Moderator
  • Hero Member
  • ****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Re: Ramblings and Updates
« Reply #10 on: February 28, 2015, 02:14:54 am »

Thanks for the update Jones, looks like you have some great plans.

1. Could you clarify point 7:
7. The transaction is signed by Run then broadcasted to the network, all is good.
How are you broadcasting to the network?

Another use for Run:

2. Let’s say that Run is a browser plugin. With current read only changes to NRS UI, more people start using trusted online public nodes as read only Nxt wallets (blockchain explorers). The existence of Run could enable people to use online nodes with full access.  If they wanted to send a transaction instead of entering passphrase into the UI (which would be prevented on public nodes), they click a button in the UI similar to ‘send to SABnzb’ that pushes the request to Run. Run then shows up a summary of the transaction to confirm, then user signs with the passphrase, once signed Run presents the user with the hex transaction bytes/RTF code and they copy and paste back into the online public node, and the transaction is broadcast to the network.

This way users can use the fully fledged official NRS UI securely anywhere in the world with only a lightweight browser plugin.

sure :)

1. In this case, Run will be connected to the internet in a non-airgapped device, for ease of use.
This will mean that Run will need to be verified by multiple parties and deemed safe, but it should be small, so it shouldn't be difficult

I can also make the signed tx bytes appear with the QR code as they do in vapor and allow people to use it offline, but that would make it a bit slower.

2. This can work with some slight changes, I could make a version of the NRS ui that allows you to enter all tx info without your passphrase, then generates a TRF string which is picked up by the plugin and highlighted, one you click it it will bring up a popup to verify, then you click "send" and the plugin can sign and broadcast the transaction for you.
Or you can do this with unsigned transaction bytes, but it would be harder to automatically target them, you would probably have to copy paste them into a tab in the plugin, then the same thing happnens.
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

allbits

  • Full Member
  • ***
  • Karma: +31/-3
  • Offline Offline
  • Posts: 221
    • View Profile
Re: Ramblings and Updates
« Reply #11 on: February 28, 2015, 02:34:44 am »

More great stuff.  Thanks Jones.

Producing an unsigned QR code might be a good option?  I might be surfing on a non-trusted computer but could add the browser plug-in so I could set up transactions as I come across them on the net.  If my phone has a trusted app to read the QR, sign the transaction and broadcast it, that would be cool.

NxtKit?
Logged

jones

  • Board Moderator
  • Hero Member
  • ****
  • Karma: +310/-8
  • Offline Offline
  • Posts: 1043
  • write code not war
    • View Profile
    • jNxt
Re: Ramblings and Updates
« Reply #12 on: February 28, 2015, 05:11:28 am »

More great stuff.  Thanks Jones.

Producing an unsigned QR code might be a good option?  I might be surfing on a non-trusted computer but could add the browser plug-in so I could set up transactions as I come across them on the net.  If my phone has a trusted app to read the QR, sign the transaction and broadcast it, that would be cool.

NxtKit?

Yes this would work, I believe that capodieci has already integrated a service that allows for mobile phones to at least scan and broadcast signed transaction bytes, if you move the signing to the phone then you then have a mobile wallet.
Logged
-- Jones NXT-RJU8-JSNR-H9J4-2KWKY

rudeboi

  • Hero Member
  • *****
  • Karma: +55/-4
  • Offline Offline
  • Posts: 633
  • Nxt Organization Member
    • View Profile
Re: Ramblings and Updates
« Reply #13 on: February 28, 2015, 10:36:14 pm »


Thanks for the update Jones, looks like you have some great plans.

1. Could you clarify point 7:
7. The transaction is signed by Run then broadcasted to the network, all is good.
How are you broadcasting to the network?

Another use for Run:

2. Let’s say that Run is a browser plugin. With current read only changes to NRS UI, more people start using trusted online public nodes as read only Nxt wallets (blockchain explorers). The existence of Run could enable people to use online nodes with full access.  If they wanted to send a transaction instead of entering passphrase into the UI (which would be prevented on public nodes), they click a button in the UI similar to ‘send to SABnzb’ that pushes the request to Run. Run then shows up a summary of the transaction to confirm, then user signs with the passphrase, once signed Run presents the user with the hex transaction bytes/RTF code and they copy and paste back into the online public node, and the transaction is broadcast to the network.

This way users can use the fully fledged official NRS UI securely anywhere in the world with only a lightweight browser plugin.

sure :)

1. In this case, Run will be connected to the internet in a non-airgapped device, for ease of use.
This will mean that Run will need to be verified by multiple parties and deemed safe, but it should be small, so it shouldn't be difficult

I can also make the signed tx bytes appear with the QR code as they do in vapor and allow people to use it offline, but that would make it a bit slower.

2. This can work with some slight changes, I could make a version of the NRS ui that allows you to enter all tx info without your passphrase, then generates a TRF string which is picked up by the plugin and highlighted, one you click it it will bring up a popup to verify, then you click "send" and the plugin can sign and broadcast the transaction for you.
Or you can do this with unsigned transaction bytes, but it would be harder to automatically target them, you would probably have to copy paste them into a tab in the plugin, then the same thing happnens.

Great stuff.

Also agree need a catchy name for Run. Nxt Signer?
Logged
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬  ▄▀▀▀▀▀▀▀▀▄  ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬●  nimirum  ●▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
▬▬▬ ◖ENDING CENSORSHIP ONLINE◗  ◖ ICO OPEN NOW◗ ▬▬▬
 

elective-stereophonic
elective-stereophonic
assembly
assembly