Tourmaline, paddling out

Test-driving release automation

Party Wave

How better to visualize Solana’s first public testnet than as a party wave? We celebrate the forthcoming release with codename Tourmaline, after the San Diego surf park where anything goes. Will the newb next to you catch that wave? Going left, going right, why not both? A crash here or there is both acceptable and expected. Come join the party!

How to join the party wave?

On Ubuntu 16.04, install solana with snap:

$ sudo snap install solana --edge --devmode

Run solana-wallet to interact with the public testnet. Request an airdrop and send @garious some tokens for posting such a swell swell pic:

$ solana.wallet testnet.solana.com airdrop --tokens=100
$ solana.wallet testnet.solana.com pay --tokens=10 --to=83astBRguLMdt2h5U1Tpdq5tjFoJ6noeGwaY3mDLVcri
$ solana.wallet testnet.solana.com balance
90

Almost public testnet

Test-driving release automation, take 3.

Erasure coding

• Add erasure coding via --features=erasure
• Rename DataReplicator to Ncp (Network Control Plane)
• Rename Race to Or in Budget language
• Much improved code documentation
• Various bugfixes
• Bump to rustc 1.26.2 patch release

Multinode testnet

The Solana team is proud to release solana-v0.6.0, codename Pipeline!

What's New?

• Multinode testnet! Very much in its infancy, but we've massively de-risked performance concerns surrounding high-throughput multinode testnets. You can now spin up validator nodes and see that the leader is able to get the blocks to them without any TPS slowdown. That was our theory and now it's a reality.

• Three software pipelines! Last month we introduced the TPU at the Solana meetup in San Francisco. The TPU is now fully implemented, as well as a new pipelined RPU to process client requests, and a TVU to validate transactions and produce a Proof of Validation. Each pipeline is built up from stages, an abstraction we built up in Rust, inspired by CPU design.

• Network Control Plane using CRDTs! Most of the building blocks to the Avalanche architecture are now implemented. The leader avoids congesting the network by breaking blocks into segments and passing them to validators. The validators then pass their segments to their peers and reconstruct the block.

• Generalized smart contracts! In the last release, we offered a minimal unnamed builtin smart contract bytecode. In this past month, we broke it out and defined a generic interface to it. As we described at the meetup, we can now light up any language that is able to safely implement that interface.

Solana's TPS Report

We massively restructured our codebase and with that work, took a performance hit that we're still working through. We're seeing consistent transaction processing rates between 140 and 170ktps. That's running with or without a validator node in place. While we're down from 250ktps last month, we're celebrating that adding validators doesn't slow the network down in the slightest. And with our new pipelining infrastructure, it's now much easier for us to do performance analysis and identify bottlenecks. We expect to be back en route to 710ktps long before our mainnet release.

What's Next?

• Public testnet! Let's see if you can knock down our network as fast as I can. :)

• Network stability features. Auto-restart, block retransmission, erasure codes, etc.

• More pipelining! We've got a hammer and aren't afraid to use it. We've identified a fourth place that would benefit from pipelining, which we'll call the NCP, the Network Control Plane pipeline. It'll benefit the TVU in the same way RPU helps the TPU - offload control messages, leaving the TVU with a speedy, linear pipeline like the TPU.

On the Radar

• More nodes! We're only making use of 2 tiers in our Avalanche architecture. We expect that to be at least 3 tiers by mainnet and with far more nodes at each tier. This is a feature where the devil is in the details and it will be implemented over several months. We'll be busy building up infrastructure dependencies like the NCP pipeline mentioned above, erasure codes, and improved asynchronous messaging.

• More smart contracts! Now that smart contracts have been generalized to an interface, we can showcase that generality by embedding a general purpose programming language.

Code Contributions

• @mvines moved all our CI to BuildKite and created an automated release process! But is there any way we can get a new bot, @solana-grimes, to correlate new features to popular surf breaks?
• @rlkelly implemented the HD wallet feature of deterministically generating private keys. He used it to make the demo's JSON file almost exactly one million times smaller!
• @CriesofCarrots added functions to make it painless to generate test ledgers. She's also working towards bringing asynchronous messaging to our thin client.
• @jackson-sandland improved error messaging with 23 commits, touching nearly every module in the codebase.
• @djKooks reviewed the README and with each issue, either corrected it or created a ticket for it.

Multi-node, multi-contract

• Generalized smart contracts
• Various multinode fixes

Multinode is coming...

First snapshot with multinode support. You can spin up leader and validator nodes to gauge how validators affect network performance. The validators reproduce the leader state, but do not yet send back their proof of replication vote to the leader. That vote is needed to measure finality times, a metric we hope to add to the final release next week.

400,000 TPS peaks

New in v0.5.0

Solana's TPS Report: now processing 250,000 transactions per second on a single-node testnet

This release once again increases transaction throughput by 700%, from 35ktps to 250ktps (with peaks up to 400ktps), over the prior release. A big thanks to @sakridge for his GPU implementation of signature verification, which can verify signatures at 870 thousand signatures per second, roughly 20% faster than what’s required for a gigabit network at full-throttle. While we applaud Stephen’s efforts, the team is now heartbroken that we won’t be able to 7x throughput again until we move to a 40 gigabit network.

Introducing Avalanche replication

We created a new ledger replication architecture that minimizes bandwidth usage, allowing the network to scale 10’s of thousands of nodes with only a small bump in finality times. We call the architecture Avalanche and while you might guess it’s named after the way data flows downward to an increasingly large set of validators, the truth is we named it after the surf break in Ocean Beach, California where back in 2005, @aeyakovenko and @garious learned to...crash.

What’s next?

We’ll continue to optimize for transaction throughput, but now that it is approaching its theoretical maximum, the Solana engineering team is turning its attention towards smart contracts and scaling the multi-node testnet. Avalanche replication is under active development and we look forward to publishing transaction rates and finality times as the network scales from 10 to 100 to 1,000 to 10,000 nodes.

Contributions

Thanks to those that contributed code since v0.4.0
@rleungx for improving the error handling of all the command-line executables
@rlkelly for helping us move towards an asynchronous client API
@djKooks for cleaning up compiler warnings

And thanks so much to the Rust team and to the community for a fantastic language and impressive libraries (especially Rayon!). We are standing on the shoulders of giants and have never felt so productive writing systems software.

7x'ed TPS again

Single-node TPS consistently over 200,000 tps, with peaks near 400,000 tps.

• Added optional GPU offloading of signature verification
• Parallelized accounting
• Added thin client subscriptions so clients can detect the number of transactions processed
• Added thick client subscriptions so validators can replicate the ledger

Before the final release, we plan to set up a small testnet on AWS and report TPS in the context of validators replicating data. Hang tight!

garious@sagan:~/projects/solana$ cat mint-demo.json | cargo run --release --bin solana-client-demo
   Compiling solana v0.5.0-beta (file:///home/garious/projects/solana)
    Finished release [optimized] target(s) in 14.97 secs
     Running `target/release/solana-client-demo`
Parsing stdin...
Get last ID...
Creating keypairs...
Signing transactions...
Done. 148.17961087247053 thousand signatures per second, 6.748566783999999us per signature
Transfering 500000 transactions in 4 batches
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Waiting for half the transactions to complete...
Transactions processed 250292
Done. 236362.20901065393 tps
garious@sagan:~/projects/solana$ cat mint-demo.json | cargo run --release --bin solana-client-demo
   Compiling solana v0.5.0-beta (file:///home/garious/projects/solana)
    Finished release [optimized] target(s) in 15.32 secs
     Running `target/release/solana-client-demo`
Parsing stdin...
Get last ID...
Creating keypairs...
Signing transactions...
Done. 148.69206818331054 thousand signatures per second, 6.725308298us per signature
Transfering 500000 transactions in 4 batches
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Waiting for half the transactions to complete...
Transactions processed 250333
Done. 206347.23171439068 tps
garious@sagan:~/projects/solana$ cat mint-demo.json | cargo run --release --bin solana-client-demo
   Compiling solana v0.5.0-beta (file:///home/garious/projects/solana)
    Finished release [optimized] target(s) in 15.8 secs
     Running `target/release/solana-client-demo`
Parsing stdin...
Get last ID...
Creating keypairs...
Signing transactions...
Done. 148.0671473461483 thousand signatures per second, 6.753692618us per signature
Transfering 500000 transactions in 4 batches
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Transferring 1 unit 125000 times...
Waiting for half the transactions to complete...
Transactions processed 327670
Done. 395587.9590925663 tps

35,000 Transactions per Second

Loom (Protocol) renamed to Solana

Loom Protocol is no more! To avoid confusion with the Ethereum-based Loom Network, we've renamed our project to Solana. Why Solana? We're proudly paying homage to the most beautiful place we've lived, Solana Beach, California. The cofounders have fond memories of surfing mushy waves at Fletcher Cove 🌊🦈🌊 and enjoying much grub and grog at Pizza Port just around the corner. Stay in touch at https://solana.io 🍺🍕🍺

New in v0.4.0

Solana's TPS Report: now processing 35,000 transactions per second

This release offers a 700% increase in transaction throughput, from 5ktps to 35ktps, over the v0.3.0 release while also laying the groundwork for a slew of new optimizations. Thanks to the new streamer functionality contributed by @aeyakovenko, input to the server is already I/O bound and able to consistently pull transactions off the wire at over 700ktps!

Introducing smart contracts

We added a highly-constrained, but surprisingly flexible form of smart contracts called spending plans to the blockchain. We can represent postdated checks, cancellations of those checks, transaction expirations, and witness signatures.

Sunny days ahead for Solana

35ktps is just the beginning and we're optimistic about our odds of improving that by another 10x in the near-term. Here's why:

Consensus with OCC (no relation to Orange County)

Solana's soon-to-come consensus mechanism, that relies on our unique Proof of History blockchain format, won't degrade performance. We're happily moving forward under the same technical constraints of a centralized system. As we approach our theoretical maximum of 710,000 transactions we'll trade throughput for availability, but that's it!

Blockchain format updated for parallel verification

One of the most exciting aspects of Proof of History is that it allows for parallel verification. We're working to extend that property to all aspects of blockchain verification, including accounting and our smart contracts. Here's some of the changes we made:

• We've moved to a token encoding where the maximum number of tokens can be represented within a signed 64-bit integer. The new format means that we can apply the highly parallel map-reduce technique to calculating balances without spilling into multi-word integers.

• We've relaxed the ledger entry format such that transactions may be added without requiring serialization via a hash from the Proof of History hash chain. That means we get to start shipping huge batches of transactions to GPU for verification. Four thousand cores at our disposal!

Contributions

Thanks so much to those that reviewed our beta release:

• @jackson-sandland for the documentation review and test-driving the Solana benchmark suite