Nxt Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

Latest Nxt Client 1.11.5 - NEW RELEASE: Ardor 2.0.3e TestNet IS LAUNCHED!

Pages: [1]

Author Topic: chaincountdown - blockchain aware countdown timer with image server  (Read 2561 times)

altsheets

  • Full Member
  • ***
  • Offline Offline
  • Posts: 231
  • check out #AAssetNXT #AltFolio and #AssetGraphs
    • View Profile
    • AssetGraphs-v2 live examples
  • Karma: +31/-1

__________________________________________________________________________
chaincountdown.py v14

Time estimation to a target block height of nxt/hz/....

Image rendered at request. Comes with httpserver.
Choose heading, fontsizes, fontface, colors.

Built for for the nxthacks voting, and beyond.
quickstart

This PNG image is generated automatically, and always up-to-date:
http://altsheets.ddns.net:8888/nxt/495997.png?heading=nxthacks%20voting

The image style can be adapted to any webpage, like here (in the center):
http://altsheets.ddns.net/assetgraphs/

license
Placed under my giveback license v05. Please give generously. Thx.

manual
Much more infos, e.g. on how to choose better colors, sizes, font, etc:
http://altsheets.ddns.net:8888

__________________________________________________________________________

TL;DR:
A Blockchain aware countdown timer !



« Last Edit: January 23, 2016, 11:19:03 pm by altsheets »
AltFolio | Newbium DataSite | AAssetNXT & -HZ | AssetGraphs | ABEE | Advice | assetparser.py & shareholders.py | bamm.py | PeerCrawler | Github e.g. ChainCountDown, ethjsre | ... much more | dividends soon paid in AAssetNXT & AAssetHZ, autumn/winter 2016

altsheets

  • Full Member
  • ***
  • Offline Offline
  • Posts: 231
  • check out #AAssetNXT #AltFolio and #AssetGraphs
    • View Profile
    • AssetGraphs-v2 live examples
  • Karma: +31/-1

New much improved version v10:

v10 Fonts & Static pages - detected automatically.

* all fontnames & static files are collected at startup
* new modules: helpers.py, imaging_tests.py
* two new image generators:
* all available fonts - 'allfonts.png' image is re-rendered when you call: python imaging.py
* all fonts with increasing size (to test minimum readable fontsize). Call: python imaging_tests.py
* new static page: errors.html with all errors that are caught

v09: all automatic now:

* at startup, prints folders, username, and current time, good for -->
* logging
* README-serverinstall.txt : starts the webserver not as root but as dedicated user



allfonts.png:


You choose your own fonts!


And *tattaa* the first donation came in! Thx!

Learn more at: http://altsheets.ddns.net:8888

:-)
AltFolio | Newbium DataSite | AAssetNXT & -HZ | AssetGraphs | ABEE | Advice | assetparser.py & shareholders.py | bamm.py | PeerCrawler | Github e.g. ChainCountDown, ethjsre | ... much more | dividends soon paid in AAssetNXT & AAssetHZ, autumn/winter 2016

altsheets

  • Full Member
  • ***
  • Offline Offline
  • Posts: 231
  • check out #AAssetNXT #AltFolio and #AssetGraphs
    • View Profile
    • AssetGraphs-v2 live examples
  • Karma: +31/-1

v11: "Conceptual Thunderstorm" - coding in the old programming language "English":

* new architecture.html
* new todo.html
* better errors.html
* github wikiwiki
* github issues
* I started this session (the 3rd after waking up 26 hours ago) in a MINDMAP software.
* What inspired me much ... to still continue, and happily... was this reaction :
   " It works! This is awesome. Tomorrow, i will make it work for ethereum. "
   Thanks! Feedback is a booster ...


v12: ... learning Docker. My very first ...

* ... Dockerfile, and
* DockerHub repository altsheets/chaincountdown
* and a lot of extra learning, because "Windows is different" again :-)
* already have an idea for an even better Dockerfile
   - one which always pulls the newest sourcecode from github. Next time ...
* oh, and actually *lol* ...
   I have won the nxthacks2015 - the voting poll of which was the inspiration for this chaincountdown.


v13: Docker magic

* learnt lots more about Docker, powerful stuff ... and applied it:
* see github chaincountdown/docker for these files, and more info on:
  * clonethenrunserver.sh is running inside the container
      * it cleverly clones the newest source code from github, before it starts the CCD server
      * and alternatively, withssh.sh additionally allows for ssh access into the docker container
  * dockerbuild.sh is building an image from the Dockerfile
  * dockercheatsheet.txt = useful docker commands
  * README.md exlaining it all
* autostarting using systemd
   * chaincountdown.service for systemd.
   * instructions how to install it.


ALL of the above, with tons of URLs linking to all the files ...
... in the LIVE VERSION of this neat and useful image server.


:-)
« Last Edit: September 02, 2015, 02:55:01 am by altsheets »
AltFolio | Newbium DataSite | AAssetNXT & -HZ | AssetGraphs | ABEE | Advice | assetparser.py & shareholders.py | bamm.py | PeerCrawler | Github e.g. ChainCountDown, ethjsre | ... much more | dividends soon paid in AAssetNXT & AAssetHZ, autumn/winter 2016

altsheets

  • Full Member
  • ***
  • Offline Offline
  • Posts: 231
  • check out #AAssetNXT #AltFolio and #AssetGraphs
    • View Profile
    • AssetGraphs-v2 live examples
  • Karma: +31/-1

version update

v13.5: Block times revisited

ChainCountDown (CCD) image server:
thread nxtforum
thread bitcointalk
github - shorturl 2020.fm/ccd
live server

__________________________________________________________________________


If you like this, show it:   NXT-CMKU-ZQYK-V6CD-9UHF4   - thanks.




Live rendered example image:


http://altsheets.ddns.net:8888/nxt/621000.png?heading=Upgrade%20your%20NXT&hea....


Please retweet this:



« Last Edit: January 07, 2016, 12:00:31 am by altsheets »
AltFolio | Newbium DataSite | AAssetNXT & -HZ | AssetGraphs | ABEE | Advice | assetparser.py & shareholders.py | bamm.py | PeerCrawler | Github e.g. ChainCountDown, ethjsre | ... much more | dividends soon paid in AAssetNXT & AAssetHZ, autumn/winter 2016

altsheets

  • Full Member
  • ***
  • Offline Offline
  • Posts: 231
  • check out #AAssetNXT #AltFolio and #AssetGraphs
    • View Profile
    • AssetGraphs-v2 live examples
  • Karma: +31/-1

EDITED heavily, to avoid repetition below. See this post for the continuation of this.


chaincountdown v18
Motivation, why all this --> A user had complained that my image server 'ChainCountDown' was broken ... but it actually was jxnt.org which was down - which is the server I had been using for API queries (of 'getBlockchainStatus' --> blockNumber). New solution: Whenever an API lookup is failing, the next server is asked. Only if all nodes are failing, my server is in trouble. But now server names are hardcoded in config.py - so I wanted to select servers which may last a while. Choosing by looking at their domain names.

-->

Domain lookup for NXT nodes and HZ nodes
PeerExplorer for HZ - updated!


Code: [Select]
mkdir -p testing; cd testing; rm peerMagic.py* -f
apt-get -y install wget python
wget https://raw.githubusercontent.com/altsheets/chaincountdown/master/tools/peerMagic.py
python peerMagic.py
Or on Windows: install Python 2.7, then download & doubleclick peerMagic.py

You can
* study the source code
* look at the NXT results
* and the -really interesting- HZ results,
* and see the network analysis results below.

---

For NXT


... I could use www.peerexplorer.com results, so the main work was a multi-threaded DNS lookup.

Plus meaningful sorting mechanisms:
* The domain names are sorted back to front, with TLD as the most significant split part.
* The IP addresses are in increasing order, with most significant the left-most byte.

For 30 of the 46 IPs with open nxt API ... I got domain names:

Code: [Select]
  94.226.61.134:                       94-226-61-134.access.telenet.be
   81.83.17.240:                           d515311F0.static.telenet.be
167.114.113.197:                                   ps12.bitcoindark.ca
167.114.113.201:                                   ps14.bitcoindark.ca
   198.27.81.25:                                         ai.lasthop.ca
   52.28.97.208:   ec2-52-28-97-208.eu-central-1.compute.amazonaws.com
   52.29.122.21:   ec2-52-29-122-21.eu-central-1.compute.amazonaws.com
   192.3.196.10:                    192-3-196-10-host.colocrossing.com
    80.86.92.70:                                  nxt.cybermailing.com
  62.75.159.113:                                vader.cybermailing.com
 198.105.223.72:                    198.105.223.72.static.midphase.com
   62.57.18.152:                         62.57.18.152.dyn.user.ono.com
 198.46.193.111:                       111-193-46-198.openvirtuals.com
 76.176.202.107:                     cpe-76-176-202-107.san.res.rr.com
  69.163.47.173:                                  host4.topserver4.com
  85.25.198.120:                               astra2991.server4you.de
  62.75.145.171:                                  euve35997.vserver.de
   178.63.60.81:            static.81.60.63.178.clients.your-server.de
 176.31.167.127:                                  127.ip-176-31-167.eu
    5.9.180.173:                                   daemon.bevolved.net
  80.150.243.95:                                     multi.djaeger.net
 167.114.113.25:                                 25.ip-167-114-113.net
167.114.113.250:                                250.ip-167-114-113.net
 167.114.113.27:                                 27.ip-167-114-113.net
  37.187.96.196:                                  humanoide.thican.net
 188.165.250.19:                                    megaman.thican.net
   82.0.250.119:    cpc80669-stap13-2-0-cust118.12-2.cable.virginm.net
   62.194.6.113:                                 h6113.upc-h.chello.nl
174.140.167.239:                               abctc.vps.nxtcrypto.org
162.243.122.251:                           jefdiesel.vps.nxtcrypto.org
   91.239.69.78:                         91.239.69.78.delfintelecom.ru
146.185.168.142:                                                nxt.sx
 91.202.253.240:                                             qob.pp.ua
 46.101.239.121
  69.163.40.132
  80.150.243.12
  80.150.243.13
  80.150.243.96
  80.150.243.97
  80.150.243.98
  80.150.243.99
  91.205.131.95
   91.235.72.49
   94.102.50.75
  95.215.47.221
 104.193.41.253
  158.69.33.172
 178.33.203.157
192.227.136.194

These are all IPs with open API!

---

For Horizon HZ

... I could not query any peerexplorer.com server, so I developed my own crawler:

(1) ask 'woll-e.net' for 'getPeers' ... answer is 659 APs (address+port, or IP)
(2) enqueue them all, and start 150 worker threads
(3) in each thread ... dequeue one AP, add it to 'apDone'.
(4) if an IP answers to a 'getTime' query on 7776, save it to 'openAPI'.
(5) if AP is answering on peer node port (usually 7774) to 'getInfo', then query 'getPeers' - then enqueue all its (yet-unvisited) peers.
(6) store all results in lists, do cool infoline printing ...
(7) keep on working through the queue ... until it is finally empty:

Ready. After checking 669 APs, on two ports each (and in ~20 seconds only!) ...:

Found
187 HZ nodes, and
63 with open API (answering on port 7776).

---

Of 63 nodes with open API, 44 had a domain name.
To see all details, scroll down in this output file peerMagic_HZ.txt.


I still did not really know how to speak to peernodeservers. Is the port 7774 communication really not documented other than so? During my first attempt,  I covered only ~2/3 of the network:

Quote
BUT that way, I could NOT find 85.214.65.220 which is api.nhzcrypto.org.
So ... something in my strategy is not perfect yet. I might not be finding all nodes!
Happy about any suggestion.

MaWo's script helped ... and after a lot of fiddling, I found the way. Yiehah.


Now ChainCountDown is more resilient, because instead of 1 server per coin, I hardcoded 6 and 7 addresses with open API. That should do the trick.

:-)


If you like this, show it:
  NHZ-Q675-SGBG-LQ43-D38L6   or   NXT-CMKU-ZQYK-V6CD-9UHF4
  or hard cash [BTC] 13whfNFT2dse7eiuNXVkuJivjxqSXHrW4j
 
EDIT: retweet


EDITED heavily, to avoid repetition below. See this post for the continuation of this.
« Last Edit: January 25, 2016, 06:33:04 pm by altsheets »
AltFolio | Newbium DataSite | AAssetNXT & -HZ | AssetGraphs | ABEE | Advice | assetparser.py & shareholders.py | bamm.py | PeerCrawler | Github e.g. ChainCountDown, ethjsre | ... much more | dividends soon paid in AAssetNXT & AAssetHZ, autumn/winter 2016

altsheets

  • Full Member
  • ***
  • Offline Offline
  • Posts: 231
  • check out #AAssetNXT #AltFolio and #AssetGraphs
    • View Profile
    • AssetGraphs-v2 live examples
  • Karma: +31/-1

chaincountdown v18

HZ peer network - analysis

Once the above crawler was ready, a natural next step was to make a network graph from the collected data.

This is how it looks:



Two nodes (green spheres) are connected with edges (gray lines) if they call each other 'peers'.
The more peers a node has (that is called the degree of a node), the bigger its sphere.

Strange shape, isn't it?

The network is very dense, so most nodes end up drawn into one big ball.
But those which are not, were catching my attention. Irregularities are interesting.

I tried an automatic clustering algorithm (i.e. "color nodes equal which are similar"),
called Louvain communities, and those nodes on the top right are clearly recognized
to be different from the all other ones:



You also see some quantitative measurements of network properties. For example:
"Path length" means how many steps a transaction needs from one node to another one.
"Diameter=4" says, it takes maximum 4 steps for a transaction to reach any other node,
and "characteristic path length" that on average only 1.769 steps.

The "Number of shared neighbors" is interesting. Many nodes are almost copies to each other.


But back to the question ... what is making those nodes at the top right so
different, that they are drawn so far out of the main bulk? Ah ... idea:
I checked for the HZ version ("NHZ V5.4") of each node, and colored them:



And now it was all clear: The nodes on the top right ... should be -but have not been- updated :-)
The mainstream (yellow) nodes do not like nostalgic, old-fashioned nodes - and do not connect to them.

The three V5 versions are mixing well, the network graph is showing that too.

Riddle solved.

By the way, here is the first picture with which I realized that I had probably hit the right reason:



The placement of the nodes is a bit artificial, it uses a "optimize inside clusters only" layouting.
But it gives immediate feedback that coloring ("NHZ versions") has something to do with graph structure.
The "NHZ 4.0e" version still has a lot of fans  ;) (RED nodes).


... Anyways, back on track:

Now I dropped all older-than-V5 nodes from the picture - and 160 IP addresses remained:



As they are hyperconnected, with many more links between them than needed
(diameter 2, average path length 1.722, average peers 44.225 / 159 = 28% of all nodes)
it is very difficult to get a meaningful automatic layout.

What I employed is the automatic partitioning "BlockModelling" by "Structural Equivalence"
... which identified 7 nodes (blue) to be different from all the other >150 ones.
They also seem to be the ones with the largest degree. I placed them manually.

At the bottom right you see a (log-log) histogram of degree versus number-of-such-nodes.

That's it, for now.


A typical next question of network analysis would be "resilience":
* ... against random failure: How many nodes have to fail until the network is fragmenting.
* ... against attacks: How many of the most connected nodes have to fail, until the network is fragmenting.

That can be examined in numerical simulations. Perhaps you want to try, then all network files are here ...


That was a really enjoyable, geeky weekend. What had started as a short & simple
user question "your image server is down" ... evolved into a whole research project.

Fascinating, no?

 ;)

If that was entertaining, enlightening, cool, or simply ... interesting
- then please now consider a way to give back. Thanks. You're good.
« Last Edit: January 26, 2016, 11:24:35 am by altsheets »
AltFolio | Newbium DataSite | AAssetNXT & -HZ | AssetGraphs | ABEE | Advice | assetparser.py & shareholders.py | bamm.py | PeerCrawler | Github e.g. ChainCountDown, ethjsre | ... much more | dividends soon paid in AAssetNXT & AAssetHZ, autumn/winter 2016
Pages: [1]