elective-stereophonic
elective-stereophonic
NRS v1.5.3e
Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Nxt Client: Nxt 1.11.15

Pages: 1 [2]  All

Author Topic: NRS v1.5.3e  (Read 8927 times)

Bernard Lerring

  • Sr. Member
  • ****
  • Karma: +30/-3
  • Offline Offline
  • Posts: 296
    • View Profile
Re: NRS v1.5.3e
« Reply #20 on: April 21, 2015, 05:42:43 pm »

Just trying out the 1.5x version for the first time. Been using 1.4x for a few months and I usually copy over "nxt_db" directory in Ubuntu to instantly have an up to date block chain.

Copying a 1.4x "nxt_db" directory over to a 1.5x installation doesn't work. Is the block chain stored in a different format?

I've stopped the client, deleted "nxt_db" and I'm letting it download naturally. I'm just curious.
Logged

abctc

  • Hero Member
  • *****
  • Karma: +148/-13
  • Offline Offline
  • Posts: 1396
    • View Profile
Re: NRS v1.5.3e
« Reply #21 on: April 21, 2015, 05:59:27 pm »

Copying a 1.4x "nxt_db" directory over to a 1.5x installation doesn't work.
- you are wrong. It works perfectly, as always (nxt_db - on mainnet).
Logged
Welcome to the Nxt generation of crypto!   Magis quam Moneta (More than a Coin)
"Do not worry, it is an attack" (c) Jean-Luc

Bernard Lerring

  • Sr. Member
  • ****
  • Karma: +30/-3
  • Offline Offline
  • Posts: 296
    • View Profile
Re: NRS v1.5.3e
« Reply #22 on: April 21, 2015, 06:05:05 pm »

Really? Maybe it copied wrong. I'll try again.

Edit: Nope. It starts eating resources and I get this:

Code: [Select]
2015-04-21 19:06:02 FINE: Database connection pool current size: 2
2015-04-21 19:06:02 FINE: Will apply sql:
DROP TABLE IF EXISTS poll
2015-04-21 19:06:02 FINE: Will apply sql:
DROP TABLE IF EXISTS vote
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE TABLE IF NOT EXISTS vote (db_id IDENTITY, id BIGINT NOT NULL, poll_id BIGINT NOT NULL, voter_id BIGINT NOT NULL, vote_bytes VARBINARY NOT NULL, height INT NOT NULL)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE UNIQUE INDEX IF NOT EXISTS vote_id_idx ON vote (id)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE UNIQUE INDEX IF NOT EXISTS vote_poll_id_idx ON vote (poll_id, voter_id)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE TABLE IF NOT EXISTS poll (db_id IDENTITY, id BIGINT NOT NULL, account_id BIGINT NOT NULL, name VARCHAR NOT NULL, description VARCHAR, options ARRAY NOT NULL, min_num_options TINYINT, max_num_options TINYINT, min_range_value TINYINT, max_range_value TINYINT, finish_height INT NOT NULL, voting_model TINYINT NOT NULL, min_balance BIGINT, min_balance_model TINYINT, holding_id BIGINT, height INT NOT NULL)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE TABLE IF NOT EXISTS poll_result (db_id IDENTITY, poll_id BIGINT NOT NULL, result BIGINT, weight BIGINT NOT NULL, height INT NOT NULL)
2015-04-21 19:06:02 FINE: Will apply sql:
ALTER TABLE transaction ADD COLUMN IF NOT EXISTS phased BOOLEAN NOT NULL DEFAULT FALSE
« Last Edit: April 21, 2015, 06:07:38 pm by Bernard Lerring »
Logged

martismartis

  • Hero Member
  • *****
  • Karma: +73/-10
  • Offline Offline
  • Posts: 1237
    • View Profile
Re: NRS v1.5.3e
« Reply #23 on: April 21, 2015, 06:14:17 pm »

Really? Maybe it copied wrong. I'll try again.

Edit: Nope. It starts eating resources and I get this:

Code: [Select]
2015-04-21 19:06:02 FINE: Database connection pool current size: 2
2015-04-21 19:06:02 FINE: Will apply sql:
DROP TABLE IF EXISTS poll
2015-04-21 19:06:02 FINE: Will apply sql:
DROP TABLE IF EXISTS vote
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE TABLE IF NOT EXISTS vote (db_id IDENTITY, id BIGINT NOT NULL, poll_id BIGINT NOT NULL, voter_id BIGINT NOT NULL, vote_bytes VARBINARY NOT NULL, height INT NOT NULL)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE UNIQUE INDEX IF NOT EXISTS vote_id_idx ON vote (id)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE UNIQUE INDEX IF NOT EXISTS vote_poll_id_idx ON vote (poll_id, voter_id)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE TABLE IF NOT EXISTS poll (db_id IDENTITY, id BIGINT NOT NULL, account_id BIGINT NOT NULL, name VARCHAR NOT NULL, description VARCHAR, options ARRAY NOT NULL, min_num_options TINYINT, max_num_options TINYINT, min_range_value TINYINT, max_range_value TINYINT, finish_height INT NOT NULL, voting_model TINYINT NOT NULL, min_balance BIGINT, min_balance_model TINYINT, holding_id BIGINT, height INT NOT NULL)
2015-04-21 19:06:02 FINE: Will apply sql:
CREATE TABLE IF NOT EXISTS poll_result (db_id IDENTITY, poll_id BIGINT NOT NULL, result BIGINT, weight BIGINT NOT NULL, height INT NOT NULL)
2015-04-21 19:06:02 FINE: Will apply sql:
ALTER TABLE transaction ADD COLUMN IF NOT EXISTS phased BOOLEAN NOT NULL DEFAULT FALSE

Updating from 1.4 to 1.5 needs updating of database with new fields in it. It takes some time, be patient. For me it took about 1 hour.
Logged

Bernard Lerring

  • Sr. Member
  • ****
  • Karma: +30/-3
  • Offline Offline
  • Posts: 296
    • View Profile
Re: NRS v1.5.3e
« Reply #24 on: April 21, 2015, 07:28:22 pm »

I thought it was maybe a new way of indexing or interpolating parts of the block chain. So, once a chain is incorporated into a 1.5x client it is different from a 1.4x chain?
Logged

toenu

  • Full Member
  • ***
  • Karma: +52/-1
  • Offline Offline
  • Posts: 231
    • View Profile
Re: NRS v1.5.3e
« Reply #25 on: April 21, 2015, 07:33:27 pm »

Just tried out the tagged data API.

What is the "type" field intended for? It seems to take arbitrary data but is not searchable.
Is it meant for file extensions/mime types, or more a general description like "video" or something else. (i.e. when using for torrents)

Also, I created data with the tags "test,asdf,1234". For some reason it takes "test" as one tag and "asdf,1234" as another.
« Last Edit: April 21, 2015, 08:15:29 pm by toenu »
Logged

martismartis

  • Hero Member
  • *****
  • Karma: +73/-10
  • Offline Offline
  • Posts: 1237
    • View Profile
Re: NRS v1.5.3e
« Reply #26 on: April 21, 2015, 07:36:51 pm »

I thought it was maybe a new way of indexing or interpolating parts of the block chain. So, once a chain is incorporated into a 1.5x client it is different from a 1.4x chain?

Let's name it rescan and optimization of database :)
Logged

Bernard Lerring

  • Sr. Member
  • ****
  • Karma: +30/-3
  • Offline Offline
  • Posts: 296
    • View Profile
Re: NRS v1.5.3e
« Reply #27 on: April 21, 2015, 07:56:18 pm »

Nice. So I take it that multiple assets/asset value etc. will load up much faster on the optimized block chain? It sometimes takes about 30 seconds on my old netbook in 1.4x. Still fast on my normal PC though.
Logged

neofelis

  • Hero Member
  • *****
  • Karma: +74/-12
  • Offline Offline
  • Posts: 568
    • View Profile
Re: NRS v1.5.3e
« Reply #28 on: April 21, 2015, 09:35:06 pm »

How do I start forging?  The "forging" link in the upper left is gone.

NVMD - figured it out. ::)
« Last Edit: April 22, 2015, 03:03:58 am by neofelis »
Logged

jabo38

  • Sr. Member
  • ****
  • Karma: +40/-38
  • Offline Offline
  • Posts: 381
    • View Profile
Re: NRS v1.5.3e
« Reply #29 on: April 21, 2015, 10:33:47 pm »

probably the wrong place to ask, but I looked and couldn't find one.  Could somebody please send me some test NXT please? NXT-2SBM-BM59-T7XC-8TEFU
Logged
Never Enough Money

EvilDave

  • Hero Member
  • *****
  • Karma: +341/-40
  • Offline Offline
  • Posts: 1789
    • View Profile
    • NXT Foundation
Re: NRS v1.5.3e
« Reply #30 on: April 22, 2015, 12:15:04 am »

@Jabo......
https://nxtforum.org/testnet/some-testnxt-to-test-asset-exchange/msg174890/#msg174890

Sending you a 1000 TestNXT, anyway, as i have more than enough for my needs. ;D
Logged
Nulli Dei, nulli Reges, solum NXT
NXT Donations: NXT-BNZB-9V8M-XRPW-3S3WD
We will ride eternal, shiny and chrome!

jabo38

  • Sr. Member
  • ****
  • Karma: +40/-38
  • Offline Offline
  • Posts: 381
    • View Profile
Re: NRS v1.5.3e
« Reply #31 on: April 22, 2015, 05:21:57 am »

Thanks!  I'll bookmark that. 
Logged
Never Enough Money

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.3e
« Reply #32 on: April 22, 2015, 10:32:05 am »

What is the "type" field intended for? It seems to take arbitrary data but is not searchable.
Is it meant for file extensions/mime types, or more a general description like "video" or something else. (i.e. when using for torrents)
I haven't decided yet actually. I intended it to be a mime type, to be used to tell the browser what to do when receiving such file. But we can also leave it free from, and if not a valid mime type default to serving it as a binary file or plain text, if isText=true, and then specific plugins can use it for other purposes.

It is not searchable by the full text search, but I can add it as an optional filter to all APIs. We could also add a separate field, "channel", if we want to group data by some more general type, e.g. news, torrents, leaks, pics, and leave the type field strictly for mime type. As long as the basic framework of uploading, searching, extending, and pruning the data is working, defining what exactly metadata fields they should have is not difficult, we just need to agree on what would be most useful.

Quote
Also, I created data with the tags "test,asdf,1234". For some reason it takes "test" as one tag and "asdf,1234" as another.
Strange indeed, but this is how the Lucene analyzer does it. It has some heuristics whether to split at punctuation or not, I believe trying to detect product numbers, hostnames, etc. To be safe, use whitespace as separator. I prefer not to mess with this, because the text processing needs to be consistent both at indexing and at searching time, and we don't have any control over the defaults used by H2 (which is also using an old Lucene version). Later we may handle search independently of H2, creating our own more customizable Lucene index.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

box1413

  • Hero Member
  • *****
  • Karma: +101/-4
  • Offline Offline
  • Posts: 687
    • View Profile
Re: NRS v1.5.3e
« Reply #33 on: April 22, 2015, 03:25:19 pm »

im curious to why the max message size is 42kb? since its going to be pruned anyway, why not set a higher limit? the pruning still it doesnt stop someone to concating multiple messages to form a bigger file.
Logged

crimi

  • Hero Member
  • *****
  • Karma: +122/-11
  • Offline Offline
  • Posts: 863
    • View Profile
Re: NRS v1.5.3e
« Reply #34 on: April 22, 2015, 05:52:04 pm »

im curious to why the max message size is 42kb? since its going to be pruned anyway, why not set a higher limit? the pruning still it doesnt stop someone to concating multiple messages to form a bigger file.

I guess total blocksize, would be at somepoint hard to target/validate the 1 min blocktime anymore.
Logged

Jean-Luc

  • Core Dev
  • Hero Member
  • *****
  • Karma: +816/-81
  • Offline Offline
  • Posts: 1610
    • View Profile
Re: NRS v1.5.3e
« Reply #35 on: April 22, 2015, 09:55:43 pm »

The total block payload size is still around 44k (which in practice can mean more than 220k when expressed as json). Increasing this risks creating problems with fork resolution, because a node may need to download multiple blocks and switch between forks before the time for the next block comes. If the transaction payload size is to be increased further, this has to be done with downloading the prunable data out of band, after the fork resolution, but this is too complicated to be done in this release.
Logged
GPG key fingerprint: 263A 9EB0 29CF C77A 3D06  FD13 811D 6940 E1E4 240C
NXT-X4LF-9A4G-WN9Z-2R322

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: NRS v1.5.3e
« Reply #36 on: April 23, 2015, 12:12:34 am »

The total block payload size is still around 44k (which in practice can mean more than 220k when expressed as json). Increasing this risks creating problems with fork resolution, because a node may need to download multiple blocks and switch between forks before the time for the next block comes. If the transaction payload size is to be increased further, this has to be done with downloading the prunable data out of band, after the fork resolution, but this is too complicated to be done in this release.
Can we have 10MB blocks?

It would be very cool to just allow for this possibility with the next hard fork. As bitcoiners debate ad nauseum about this, NXT can just do it.

I doubt any block will fill up so it would just be a theoretical limit. It does concern me that a single bot can totally fill up the current block via InstantDEX arbitrages

James

P.S. We can even use 1MB (or 1.7MB) blocks as the equivalent of 10MB BTC blocks due to blocktime difference
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

ScripterRon

  • Hero Member
  • *****
  • Karma: +75/-2
  • Offline Offline
  • Posts: 523
    • View Profile
Re: NRS v1.5.3e
« Reply #37 on: April 23, 2015, 12:43:03 am »

Can we have 10MB blocks?
This is something I ran into as I was implementing the new WebSocket support.  It turns out that NRS needs to be able to download up to 719 blocks at a time in order to handle fork resolution.  So the maximum response message size becomes the upper limit on the average block size unless the fork resolution code is redesigned.  Or as JL said, non-essential data needs to be downloaded separately to reduce the message size.
« Last Edit: April 23, 2015, 12:45:10 am by ScripterRon »
Logged

jl777

  • Hero Member
  • *****
  • Karma: +718/-123
  • Offline Offline
  • Posts: 6170
    • View Profile
Re: NRS v1.5.3e
« Reply #38 on: April 23, 2015, 01:06:44 am »

Can we have 10MB blocks?
This is something I ran into as I was implementing the new WebSocket support.  It turns out that NRS needs to be able to download up to 719 blocks at a time in order to handle fork resolution.  So the maximum response message size becomes the upper limit on the average block size unless the fork resolution code is redesigned.  Or as JL said, non-essential data needs to be downloaded separately to reduce the message size.
How often does it need to get all 719 blocks? if it is rare maybe we can live with a theoretical slowdown. I am talking about real 1 MB blocks full of transactions. If we have to hard fork for this, might as well do it now.

James
Logged
There are over 1000 people in SuperNET slack! http://slackinvite.supernet.org/ automatically sends you an invite

I am just a simple C programmer

toenu

  • Full Member
  • ***
  • Karma: +52/-1
  • Offline Offline
  • Posts: 231
    • View Profile
Re: NRS v1.5.3e
« Reply #39 on: April 23, 2015, 06:14:01 am »

What is the "type" field intended for? It seems to take arbitrary data but is not searchable.
Is it meant for file extensions/mime types, or more a general description like "video" or something else. (i.e. when using for torrents)
I haven't decided yet actually. I intended it to be a mime type, to be used to tell the browser what to do when receiving such file. But we can also leave it free from, and if not a valid mime type default to serving it as a binary file or plain text, if isText=true, and then specific plugins can use it for other purposes.

It is not searchable by the full text search, but I can add it as an optional filter to all APIs. We could also add a separate field, "channel", if we want to group data by some more general type, e.g. news, torrents, leaks, pics, and leave the type field strictly for mime type. As long as the basic framework of uploading, searching, extending, and pruning the data is working, defining what exactly metadata fields they should have is not difficult, we just need to agree on what would be most useful.

Files could be stored as a Data-URI, which already contains the mime type. (e.g. "data:application/x-bittorrent;base64,ZDg6YW5ub3VuY2U....")
This is very practical as it can just be opened in the browser. In that case type could be omitted or used for something else.

For categorization of content in addition to tags it would definitely be useful to have type / channel as a search attribute.
« Last Edit: April 23, 2015, 06:37:50 am by toenu »
Logged
Pages: 1 [2]  All
 

elective-stereophonic
elective-stereophonic
assembly
assembly