Development Update #44

This is over view of current development status and priorities

Consensus

Consensus is being worked on. The earlier algorithms are being implemented, bench-marked. Then testing them against the newer algorithms, so we have comparison. This is currently in python. Will be moved to golang and integrated as soon as its ready.

This is not required for the distribution , because we can use central signing key until decentralized consensus is implemented.

Coin

The coin is mostly done. There are only a very few changes.

Right now, visor is managing wallets and the json API daemon is inside of daemon. We want to move the wallet management and json daemon into its own library. The webwallet interfaces through JSON API daemon and the daemon hooks in to visor through function calls. We want the json API to be very close to Obelisk/Dark Wallet. We believe the Dark Wallet API is better than the Bitcoind API.

Once consensus is done, it will act as a state controller for visor.

In Bitcoin, loading wallets takes too long, because the whole blockchain needs to be loaded from scratch and the whole history scanned. Skycoin uses a leaner interface. When a wallet is loaded, the addresses are checked for unspent outputs associated with the address. So checking current balance is instant. The API will also allow you to check balances for addresses, without have the private keys loaded. There are many things you cannot do in bitcoind (like check balances for address easily) and we think Obelisk/Dark Wallet has the best API for developers and want to move away from bitcoind.

The local client and remote/thin clients will be on same API. You can run APIs for balances or injection transactions on a local or remote server just the same. So a cell phone who does not have a copy of the blockchain, but has a wallet file can query balances or inject transactions.

History is not implemented right now. It will likely be a separate library on top. It is outside of the core, needed to run the blockchain. The blockchain on disc will have the blocks and will have copy of unspent output set at interval. Then it can rewind back a month to an unspent output set, then apply the blocks in that month in order until the next checkpoint. So we can load wallet histories going from most recent transactions, going farther back in time.

This means wallets will load instantly, balances will be available instantly, coins can be sent instantly and history (when implemented) will load in background. As the Bitcoin blockchain gets longer, loading wallets will continue to slow down.

Skycoin already has multiwallet support, so you can load/unload multiple wallets in the client. Importing deterministic wallets from keys need to be added to the gui, after the wallet internals and refactor is done.

Bitcoin used Berkeley Key value store for serialization format of the wallet and there was problems. As the library was updated, only version wallets from very early Bitcoin versions, appeared empty when loaded. The old serialization format could not be loaded by the new client. So a wallet may say it is empty, but actually contain 100,000 Bitcoin. You have to manually extract the private keys with a tool and then reimport them on the command line. This caused some confusion and frustration.

Skycoin is using a JSON wallet format. All wallets are deterministic and the generation key will be in the wallet file. As long as you have the wallet seed, you can recover the private keys and coins. The internal format of the JSON wallet is something we have not had time to think through and finalize, but we can safely nuke everything in the wallet file except the wallet seed without risk of losing coins when the wallet storage format is upgraded.

In Bitcoin using unspent output set snapshots was a security risk, because there was no way to validate them without starting from the genesis block and applying every block since genesis. In Skycoin, there is a rolling hash of the operations applied to the blockchain state, so if you have two checkpoints and apply the blocks between the checkpoint to the first checkpoint, you will get the second checkpoint. This means that the blockchain can be divided into check point intervals and each interval can be validated in parallel. So you can validate backwards and can validate forwards, where as Bitcoin only lets you validate forwards.

The system is designed that way, so eventually you will not need to have the whole blockchain on every node. A node only needs a trusted copy of the unspent output set and then every block since that checkpoint. Then it can check balances and inject transactions.

We are evaluating this. We will probably have two hashes. One is the XOR of the SHA256 of each unspent output at the beginning of the block. This is weak hash, but has constant time to update. In theory, someone can add ~300 unspent outputs to the set and find a subset that generate the same same hash for this. We are not relying on this for validation, but it is good for sanity checking.

Checkpoints of the unspent output set need to have a full merkle-tree hash of the SHA256 of the serialization of each unspent output and have to be tied to a particular block header. The checkpoint needs to be signed by a public key of person who vouches for it. If check points are enabled, you would put in public key for ~8 trusted nodes (exchanges, friends, maybe dev key). It would download the 8 copies of the checkpoint hash from each person and make sure they all agree. If two different people in the trusted set have different hashes for the serialized unspent output set at a given checkpoint, then client should go into a warning/emergency mode until it is resolved by user.

The second has is a merkle tree root of the SHA256 of the serialization of the consumed/produced outputs from the block, accumulated into the hash from the previous block. This makes the block header, not only a function of the previous block input, but also a function of the “state” of the blockchain (the unspent output set) when the block was applied. There are three levels - block header is only function of previous block header and transactions (data in the block), (Bitcoin does this) - block header is a function of previous block header, contents of current block and subset of “state” the block is being applied against (Skycoin)

We are trying to determine if this adds anything or if it is redundant. If two clients have different internal state, such as an output being created and then having bit flipped randomly in the balance by hardware failure, this will detect the divergence in the next block if the affected data is an input (consumed) or output (created by) of the transaction. This will also detect an error, where a 32 bit system or 64 bit system output different values (rounding errors, optimization, different versions of compiler).

We can catch these errors in the next block, where as in Bitcoin they may persist many blocks until the unspent output is used and then cause a fork. For instance, this method guards against unintentionally introduce rounding errors and protects against the Block Index 0 fork, if Satoshi tried to spend the reward on block 0. We catch the non-determinism in the output set, in the next block.

If two people apply the same block against different states, the header of the accumulator hash in the next block will be different (ideally). Ideally, you would compute a hash of the full state, each block. However, we could not find a fast enough updating function for this and felt a full merkle tree hash each block would slow down block validation too much. The best compromise, is to check the full merkle-hash at the checkpoints. The full merkle-hash will also detect any single bit-errors, in any outputs that have not been spent yet.

Coin Development Priorities

Coin Distribution Requirements:
Longer Term:

Skywire Development Priorities

Skywire has multiple components and it was not clear how things should be structured.
Skywire Advanced/Not Essential
DHT
Merkle-Dag
Goal:

Skywire Meshnet Development Priorities

Our basic 1st generation target for the software defined radio is:

We need to be able to set the amplitude of a sin wave, for the positive and negative half of the sin wave at 700 Mhz. At 700 Mhz with 8-bit of quantization, using amplitude modulation at 700 Mhz, we can transmit 11.2 Gb/s. This is the target, but we may only be able to do 300 million values per second without upgrading to ASICs. The change of amplitude must occur at the zero crossing to avoid snapping.

There is ringing in antenna and other factors. So we will probably be using the same amplitude for positive and negative portion of the carrier at start. The amplitude value will change slowly over the cycles. Lower resolution in the DAC is acceptable for prototype, but objective is setting amplitude 1.4 billion times per second at the zero crossing and at-least 8 bits of resolution in output.

For the radio module, we need to determine:
Notes:
For the antenna:

The signal attenuation at 2.4 Ghz for going through a single 5” wall, is 23 dB. Every 3 dB is 50% loss of power. Every 6 dB is 50% loss of voltage.

The signal attenuation at 700 Mhz, for going straight through shopping mall sized concrete building is only 18 dB. The range can be miles and still achieve extremely high speeds. However, if a large number of devices with unidirectional antennas are deployed, with competitive transmission power, then the same congestion and noise floor problems occur as on 2.4 Ghz. However, it is slightly different because almost none of the energy between transmitter and receiver on 2.4 Ghz is on a line-of-sight path, its mostly multipath interference. Simulation shows that viable high density meshnet will need directional antennas (120 degree coverage maximum) and needs pencil beams.

Directional beams allow very important simplifying assumptions

The first generation equipment will be a single output or perhaps two. It will be working out bugs and driving down cost. We chose a very modest technology and encoding, that minimizes technical risk. This should allow deployment rapidly. There is a very short lead-time between someone figuring out an opensource design and when commercial manufacturing/assembly/distribution of PCB boards.

The HackRF was built by people who had no experience in electronics. They were able to learn everything from Google and build a software defined radio that goes between 0 hz and 6 Hz, with 20 Mhz bandwidth. With no experience they were able to design a $160 SDR that is more powerful than $15,000 commercial SDRs. That is very encouraging. To bring the meshnet into the physical world, will require multiple teams working on different parts of the problem.

The AM encoding does not interfere with the users of the digital television band. The FM receivers will not even lock on to it. The gain for the directional beams is 20 dB to 30 dB on the receiver and transmitter. The interference produced is below level of spurious emissions from cell phones, computers and wireless telephones. A horizontal beam polarization yields 30 dB attenuation for digital television, receivers. The beam should be horizontally polarized whenever possible.

Second generation meshnet may have ~16 separate antenna elements on a chip for doing digital delay lines and driving phased arrays. The signal may be a directional gaussian type ultra-wideband pulse, with 700 Mhz pulse frequency. The AM modulated signal with 8-bit ADC caps out at 11.2 Gb/s. The ultra-wideband Gaussian modulation has extremely low level emissions across a very large spectrum and can reach theoretical rate that are measured in terabits/second. However, the technology is new and encoding 1024 bits per pulse, will require more complicated electronics, special antennas, novel encodings and engineering work. Even if the bandwidth rate is not improved, the technology is low probability of intercept and below the noise floor a few meters off the beam axis, which is desirable for operation in particular countries.

Translation bounty: 35 SKY (3182 words)

Discuss this post on telegram

Skycoin Telegram