I'm sharing everything I'm learning to become a Blockchain Engineer
Day 1 |
|---|
| Blockchain |
| - Bitcoin |
| - Ethereum |
| - Smart Contracts |
| - Bitcoin Vs Ethereum |
| - Oracle Problem & Solution |
| - Hybrid Smart Contracts |
| - Chainlink |
| - Importance of Ethereum |
| - Chainlink features |
| - Summary |
| Features of blockchain and smart contracts |
| - Decentralized |
| - Transparency & Flexibility |
| - Speed & Efficiency |
| - Security & Immutability |
| - Removal of Counterparty risks |
| - Trust Minimized Agreements |
| - Summary |
| DAOs |
| Ethereum Transaction On a Live Blockchain |
| - Wallet Creation |
| - Etherscan |
| - Multiple Accounts |
| - Mnemonic , Public & Private keys |
| - Mnemonic vs Private vs Public keys |
| - Mainnet & Testnets |
| - Initiating our first Transaction |
| - Transaction details |
| - Gas fees, Transaction fees, Gas limit, Gas price |
| - Gas vs Gas price vs Gas Limit vs Transaction fee |
| - Gas estimator |
| How Blockchain works/whats going on Inside the Blockchain |
| - Hash or Hashing or SHA256 |
| - Block |
| - Blockchain |
| - Decentralized/Distributed Blockchain |
| - Tokens/Transaction History |
| - Summary |
Day2 |
| Signing and Verifying a Transaction |
| - Public & Private Keys |
| - Signatures |
| - Transactions |
| - Summary |
| Concepts are same |
| - Nodes |
| - Centralized entity vs Decentralized Blockchain |
| - Transactions are Listed |
| - Consensus |
| - proof of work/Sybil resistance mechanism |
| - Blocktime |
| - Chain selection rule |
| - Nakamoto consensus |
| - Block Confirmations |
| - Block rewards & transaction fees |
| - Sybil attack |
| - 51% attack |
| - Drawbacks of pow |
| - proof of stake/sybil resistance mechanism |
| - Validators |
| - pros & cons of pos |
| - Scalability problem & Sharding solution |
| - Layer 1 & Layer 2 |
| - Rollups |
| - Summary |
Day 3 |
| Solidity |
| Lesson 1 - Remix IDE & its features |
| - Solidity version |
| - Defining a Contract |
| - Variable types & Declaration |
| - Solidity Documentation |
| - Initializing |
| - Functions or methods |
| - Deploying a Contract |
| - Public , Internal , private , External Visibility |
| - Modifying a Variable |
| - Scope |
| - View functions |
| - Pure function |
| - Structs |
| - Intro to storage |
| - Arrays |
| - Dynamic array |
| - Fixed array |
| - Adding to an array |
| - Compiler Errors |
| - Memory Keyword |
| - Storage keyword |
| - Mappings Datastructure |
| - SPDX license |
| - Deploying to a live network |
| - Interacting with deployed contracts |
| - EVM |
| - Summary |
Day 4 |
| Lesson 2 - StorageFactory |
| - Factory pattern |
| - New contract StorageFactory |
| - Import 1 contract into another |
| - Deploy a Contract from a Contract |
| - Track simple storage contracts |
| - Interacting with Contract deployed Contract |
| - Calling Store & Retrieve Functions from SF |
| - Address & ABI |
| - Compiling & storing in SS through SF |
| - Adding Retrieve Function |
| - Compiling |
| - Making the Code lil bit Simpler |
| - Additional Note |
| - Inheritance |
| - Recap |
Day 5 |
| Lesson 3 - Fund me |
| - purpose of this contract |
| - Payable function , wei , gwei & ether |
| - Mapping , msg. sender , msg.value |
| - Funding |
| - ETH -> USD /conversion |
| - Deterministic problem & Oracle solution |
| - Centralized Oracles |
| - Decentralized Oracle Networks |
| - Chainlink Datafeeds |
| - Chainlink Code documentation on ETH/USD |
| - Importing Datafeed code from Chainlink NPM package |
| - Interfaces |
| - ABI/Application Binary Interface |
| - Interacting with an Interface Contract |
| - Finding the Pricefeed Address |
| - Deploying |
| - Getprice function |
| - Tuples |
| - Typecasting |
| - deploying |
| - Clearing unused Tuple Variables & Deploying |
| - Making the contract look Clean |
| - Wei/Gwei Standard (Matching Units) |
| - getting the price using Get conversion rate |
| - deploying |
| - Safemath & Integer Overflow |
| - Libraries |
| - Setting Threshold |
| - Require statement |
| - Revert |
| - Deplying & Transaction |
| - Withdraw Function |
| - Transfer , Balance , This |
| - Deploying |
| - Owner , Constructor Function |
| - Deploying |
| - Modifiers |
| - Deploying |
| - Resetting the Funders Balances to Zero |
| - For loop |
| - Summary |
| - Deploying & Transaction |
| - Forcing a Trasacttion |
Day 6 |
| Lesson 4 - Web3. py SimpleStorage |
| - Limitations of Remix |
| - VScode , Python , Solidity Setup |
| - VScode features |
| - Testing python install & Troubleshooting |
| - Creating a new folder |
| - SimpleStorage. sol |
| - Remember to save |
| - VScode Solidity Settings |
| - Python Formatter & settings |
| - Author's recommended Settings |
| - working with python |
| - Reading our solidity file in python |
| - Running in Python |
| - Keyboard Shortcuts |
| - Py-Solc-x |
| - Importing solcx |
| - Compiled_sol |
| - Bracket pair colorized |
| - pysolcx documentation |
| - Printing Compiled_sol |
| - Comparison wih remix (Lowlevelstuffs , ABI) |
| - Saving Compiled Code/writing |
| - import Json |
| - Json formatting/settings |
| - Deploying in Python (Bytecode , ABI) |
| - Which Blockchain/Where to deploy |
| - Ganache Chain |
| - Ganache UI |
| - Introduction to Web3. py |
| - pip install web3 |
| - import web3 |
| - Http/Rpc provider |
| - Connecting to Ganache(RPC server,Documentation,Chain ID,address,Privatekey) |
| - Deploy to Ganache |
| - Building a Transaction |
| - Nonce |
| - Getting Nonce |
| - Create a Transaction |
| - Transaction Parameters |
| - Signing Our Transaction(signed_txn) |
| - Never Hardcode your Private keys |
| - Environment Variables |
| - Setting Environment variables |
| - Limitations of Exporting Environment Variables |
| - Private key PSA |
| - Accessing Environment Variables |
| - .env file, .gitignore, pip install python-dotenv |
| - load_dotenv() |
| - Sending the signed Transaction |
| - Deployment |
| - Block confirmation(wait_for_transaction_reciept) |
| - interact/work with thee contract |
| - Address & ABI |
| - Retrieve() , Call & Transact |
| - Store function |
| - Creating Transaction(Store_transaction) |
| - Signing Transaction(signed_store_txn) |
| - Sending Transaction(send_store_tx,tx_receipt) |
| - Deployment |
| - some nice syntax & deployment |
| - ganache-cli |
| - install Nodejs |
| - install yarn |
| - Run ganache cli , ganache documentation |
| - update privatekeys,addresses,http provider |
| - open new terminal & deploy |
| - deploy to testnet/mainnet |
| - Infura, Alchemy |
| - Create project |
| - update the rinkeby url, Chain id , address & private key |
| - Deploying |
| - summary/recap |
Day 7 |
| Lesson 5 - Brownie Simple Storage |
| - Brownie Intro & Features |
| - install Brownie |
| - 1st brownie simplestorage project |
| - Brownie Folders |
| - brownie compile & store |
| - brownie deploy |
| - brownie runscripts/deploy. py & default brownie network |
| - brownie Advantages over web3. py in deploying |
| - getting address & private key using Accounts package |
| - add default ganache account using index |
| - add accounts using Commandline |
| - add accounts using env variables |
| - adding wallets in yaml file and updating in account |
| - importing contract simplestorage |
| - importing & deploying in brownie vs web3. py |
| - recreating web3 .py script in brownie |
| - tests |
| - Deploying to a Testnet |
| - Brownie console |
Day 8 |
| Lesson 6 -Brownie Fund Me |
| - Initial Setup |
| - Dependencies |
| - Remappings |
| - Rinkeby Deploy Script |
| - init.py |
| - Deploying to Rinkeby |
| - Deploying to ganache |
| - Mock |
| - Constructor Parameters |
| - Refactoring |
| - Deploying to a persistant ganache |
| - Adding network to brownie |
| - Withdraw and Fund |
| - Testing |
| - Default Network |
| - Mainnet fork |
| - Custom mainnet-fork |
| - Adding to Github |
| - Where should I run my tests? |
Day 9 |
| Lesson 7: SmartContract Lottery |
| - Decentralized Lottery |
| - Setup |
| - Wait..is this really decentralized? |
| - Could we make it decentralized? |
| - Chainlink Keepers |
| - Lottery.sol |
| - Main functions of Lottery.sol |
| - address payable[] |
| - require minimum payment |
| - getEntranceFee |
| - Testing |
| - Enum |
| - startLottery |
| - Randomness |
| - pseudorandom numbers |
| - True Randomness with Chainlink VRF |
| - Oracle Gas and Transaction Gas |
| - Request and Receive |
| - Kovan |
| - Gas Estimation Failed |
| - After Link transaction completes |
| - After transaction completes |
| - Request and Receive |
| - Asynchronous 2 Transactions |
| - Clarification |
| - endLottery |
| - returns(type variableName) |
| - fulfillRandomness |
| - override |
| - Modulo(Mod Operation %) |
| - Lottery Testing |
| - deploy_lottery.py |
| - get_account refactored |
| - contract_to_mock |
| - Contract.from_abi |
| - Adding vrfCoordinatorMock |
| - Start Lottery |
| - enter_lottery |
| - end_lottery |
| - Funding with LINK |
| - brownie interfaces |
| - waiting for callback |
| - Integration Tests Vs Unit Tests |
| - Unit Testing |
| - pytest.skip |
| - test_cant_enter_unless_started |
| - test_can_start_and_enter_lottery |
| - test_can_end_lottery |
| - test_can_pick_winner_correctly |
| - Events and Logs |
| - callBackWithRandomness |
| - Integration Test |
| - Test deployment |
| - Topics |
| - Conftest.py |
Day 10 |
| Lesson 8: Chainlink Mix |
Day 11 |
| Lesson 9: ERC20s, EIPs, and Token Standards |
| - Why make an ERC20? |
| - How do we build one of these tokens? |
| - Solidity 0.8 |
| - 1_deploy_token.py |
Bitcoin
Bitcoin was one of the first protocol to use this revolutionary technoloy called Blockchain.Bitcoin's white paper was released by the sudo anonymous Santoshi Nakamoto and it outlined how bitcoin can be used to make peer to peer transactions in a decentralized network.This network is powered by Cryptography.It allows people to engage in censorship resistant finance in a decentralized manner.Due to some of the features of bitcoin a lot of people took it as a superior store of value over another asset like gold.That's why it is commonly referred to as digital gold.Similar to gold there's a scarce and set amount of it on the planet and people use it to buy and sell similar to other assets.You can read more about the original vision in the white paper.
Ethereum
Some people saw this technology and thought that they could do even more.A few years later a man named Vitalik Buterin released a white paper describing a new protocol called Ethereum which used the same blockchain infrastructure but with an additional feature and in 2015 they released this project called Ethereum.Him and a number of others co-founders took this blockchain technology and applied it in ways that people can make entirely decentralized applications, decentralized organizations and build smart contracts and engage in agreements without a third-party intermediary or centralized governing force.Their idea was to take the same pieces that made bitcoin great and add smart contracts to it.Technically this was not even a new idea.Back in 1994, a man names Nick Zabo proposed a technology called Smart Contracts.
Smart Contracts
A Smart Contract is a self executing set of instructions that is executed without a third party intermediary.They come to life on a blockchain and these smart contracts are really going to be a core thing that we're going to be working with and we're going to be developing.Smart contracts are similar to regular traditional contracts that people make between eachother but instead of writing these contracts down on pen and paper or typing that on the computer, it's entirely written in code.The terms of the agreement are written in code and automatically executed by the decentralized blockchain network.This is the main differentiators between the Ethereum protocol and the Bitcoin protocol.
Bitcoin Vs Ethereum
Technically Bitcoin also has smart contracts however they're not touring complete: they don't have the full range of capabilities as a turing complete application like ethereum.This is actually an intentional move by bitcoin developers.They view the bitcoin network as a asset whereas the ethereum and ethereum developers viewed that asset as an asset and also as a utility for people to build these smart contracts.These smart contracts are revolutionary technologies but they actually come with a fatal flaw with what's know as The Oracle Problem.
Oracle Problem & Solution
These blockchains are deterministic systems and we'll learn very soon.This determinism means they are a walled guarded meaning that everything that happens in these smart contracts and on this blockchain happens in this little box.Now ofcourse if you want these smart contracts to actually be these digital superior agreements then they need some way to interact with the real world and get real data and external outside the blockchain computation.This is where oracles come into play.Oracles are the devices that bring data into a blockchain or execute sometype of external computation.Great so oracles are the solution.Now blockchain can talk to real world right? Well not quite.Our blockchains and smart contracts are these decentralized applications and in order for them to stay decentralized that means they also need to get their data and external computation from a decentralized manner as well.Your on-chain logic will be decentralized on the blockchain but you'll also need your off-chain data and your external computation decentralized as well.
Hybrid Smart Contracts
Combining on-chain logic settlement layers and these off-chain data and external computation builds are called Hybrid Smart Contracts.A large majority of defi applications in the largest applications today are these hybrid smart contracts.This is where protocol chain link comes into play.
Chainlink
Chain link is a decentralized modular oracle network that allows you to bring data into your smart contracts and do external computation and these hybrids smart contracts can have this on-chain settlement and interact with the real world in some meaningful way.Chain link is an incredibly powerful oracle network because it allows us to get data, get randomness, do some type of update or really customise our smart contracts in any way we want and elevate them to do anything that we want them to do.
Importance of Ethereum
Decentralized application is usually a combination of several smart contracts.Smart contracts are going to be what we code, write and deploy for the majority in this lesson.The ethereum protocol has given rise to many new paradigms and industries including DEFI, NTFs, DAOs(decentralized autonomous organizations) , Layer2 and more.A couple of other protocol have taken this ethereum vision and gone in a different direction with it like polygon, polka dot or avalanche.If we learn the core basics of smart contract development on the ethereum platform all these skills translate to these other chains as well.
There're some smart contracts platform aka blockchains that don't use solidity however learning fundamental skills here will still translate to every single other blockchain.
Chainlink features
Chainlink is the most popular and powerful decentralized oracle network and is going to be the one we're going to focus on here.Chainlink is also blockchain and smart contract platform agnostic meaning it'll work on Ethereum, Avalanche, Polygon, Polkadot or really any blockchain or smart contract platform out there.
Summary
- Bitcoin was the first one to take blockchain mainstream
- Bitcoin is like a digital gold.
- Ethereum allows for smart contracts.
- Chainlink provides data and external computation to smart contracts.
- Hybrid smart contracts combine on-chain and off-chain computations.
Decentralized
There's no centralized source that controls the blockchain.The individuals that make up the blockchain are know as node operators.They are the independent individuals running the software that connects the whole blockchain together.It's all these different independent individuals that make the blockchain and blockchain like networks decentralized.This is so fundamentally groundbreaking is if you go back to what happened with robinhood and gamestop.Gamestop shares are no longer been allowed to be bought because a centralized entity didn't want them to be bought anymore.So they flipped the switch and nobody could buy that stock anymore.Essentially having a single entity controlling the entire financial market.The fact that single entity has the power to make these choices for us is a travesty and blockchain is here to solve that.There's a narrative here called the bankless narrative where users could actually live in a world where they don't have a bank.Bank while good in their own right have a history of doing some shady things.They also have the power to potentially freeze your funds not letting you withdraw or move or do anything because they are centralized entity that again can flip a switch and control how you interact with your life everyday.Being free of these centralized entities have this much power and this much control over your life has widespread positive remifications.
Transparency & Flexibility
Everything that's done on blockchain and all the rules that are made can be seen by everyone.There's no backdoor deals, no shady happenings.This means there's no special information that a few have.Everyone has to play by the same rules and everyone can see what those rules are.Additionally this doesn't mean that everything you do is tracked.The blockchain is pseudo-anonymous.So you can create different accounts and can interact with it in many different ways.
Speed and Efficiency
Have you ever tried to withdraw from the bank and it took 1-2 hours? All the bank is doing is adding and subtracting numbers; basic first grade math.So why does it take so long? Because blockchains are verified by a decentralized collective, the settlement or withdrawl period in this case is substantially faster and depending on the blockchain that you're using can be from 10mins all the way down to just a couple of seconds.In the stock trading or hedge fund world, it can actually take upto a week for your buy or sell of a stock to go through.
Security and Immutability
Blockchains are immutable which means they can't be changed and because of this they can't be tampered with or corrupted in any way shape of form.This allows us to have massive security on our data, transactions and anything of the like.If your computer goes down and your backup computer goes down, in the regular world your data is gone.On a blockchain if several nodes goes down, it doesn't matter because as long as one node and the entire system is running, the data is safe and secure.There are 100's of thousands of nodes running these blockchain software meaning that everything that happens, happens and is immutable and won't change.Hacking the blockchain is nearly impossible and sustantially harder than hacking a centralized entity.This is also much more secure in the asset sense as well.Instead of having a gold in a vault or contract written in a piece of paper or on your computer, you've asset that is locked on blockchain forever and all you need to do to access it is have a private key or mnemonic which is essentially a password.
Removal of counterparty risk
Smart contract in particular remove a massive conflict of interest.In the traditional world when we engage with users or individuals, they don't always have our best interests at heart.A lot of them usually self motivated in some sense and there's nothing wrong with that.However when we make an agreement with them, this agreement can have a massive conflict of interest with the user who's supposed to execute that agreement.Let's take insurance for example If I pay an insurance provider 100 dollars a month, I'm paying them a 100 dollars and in the event that I get hit by a bus, we've made an agreement or a contract that they're going to pay my medical bills or bail me out.However they have this massive conflict of interest insurance companies aren't in the business of giving out money, they're in the business of making money.So even though they signed this agreement when this event occurs, they still don't want to pay this money out to me and if they can find the loophole in the contract.They will because that is what they are motivated to do.So they sign this agreement but it's not in the best interest to do so.So they have this massive conflict of interest and this is native in all of the agreements that we make today.They are the ones who decides whether or not they're going to execute their agreement.Giving execution power to the party that doesn't want to execute something that has often led to frustration.Now the follow-up is you can always sue them and go through this process but now you're wasting all this time going through all this long process to get something that you should have originally gotten in the first place.This leads me to one of the biggest value adds to smart contracts.
Trust Minimized Agreements
Smart contracts allows us to engage in trustless and trust minimized agreements.We currently live in a world of brand based agreements.If I engage in some agreement and I don't like the service that I'm provided, my alternative to this is to waltz down the street to another brand to another service who's going to make the exact same set of promises to me and then I have to trust them that they're going to execute faithfully.Smart contracts allow us to move from this brand based agreements to math based agreements.These math based agreements we don't even have to trust that they're going to do the right thing.Hence the name trustless.One plus one always going to equal two in a math world.Whatever the code determines is the input/output that's exactly what's gonna happen every single time.These really all add up to two major pieces : freedom and trustless.All these pieces allows us to live in a world that's more accountable, more trusting, more friendly and just better it allows us to work in an environment and a universe where things just work.It allows us to do the freedom to engage with other people how we wish because there's no centralized controlling body influencing every action that we make.All the rules are the same and nobody's getting special treatment.This bring out the new world of economic oppourtunity as well and as our lives become more and more digital where constantly being bombarded with centralized services that want us to use their interface so they can profit on how we interact and force us or push us to making the decisions that they're motivated for us to make.Smart contracts, decentralized applications and blockchain allows us to be free of these repressors and live in an environment that's truly free and trustless.
Summary
- Blockchains are decentralized.They aren't controlled by a single centralized entity.
- Blockchains are transparent.Everything that happens on a blockchain everybody else can see, work with and see that everyone's playing by the same rules.
- Blockchains are quick and efficient especially when it comes to monetary policy.
- Blockchains can't be changed or tampered with or corrupted and are incredible secure.
- Smart contracts remove massive conflict of interest traditional agreements have.
- Smart contracts allows us to engage in trustless and trust minimized agreements.
- Smart contracts are typically paired with some type of oracle to get the information about the real world.When they're paired with oracle, they're called
Hybrid Smart Contracts.
DAOs
DAOS are organizations that live online and live in the smart contracts.They're similar to regular organization in the traditional world however thay have people who maybe hold governance tokens to make voting decisions or they do all the governance on chain on this decentralized settlement layer giving us the freedom to engage with eachother as we please.
First thing that we're going to need is an ethereum wallet.So go to metamask because it's one of the most popular wallet and one of the easiest to use.Download the wallet and it's going to be a little extension in the top right corner of your browser.This way we can really easily see at any time what we have in our wallet.This will store all our ethereum based currencies.Go ahead and install metamask.
Wallet Creation
We're going to create a brand new wallet so go head and hit create wallet.If you already have a wallet, you can actually import it via seed phrase.
Now we'll create our password and make sure this is really secure.If you want to use an account and actually put real ethereum and put real money into it, you absolutely need to back it up.
This is just a demo however if you show this secret phrase to anybody else they will have access to your all funds.So everything that we're gonna do, we're gonna use fake money.
Hit confirm and it gives couple of tips.Remember definitely take the tips seriously especially if you're gonna use the account for real money.
We can see the user interface of this wallet and if you look in your extensions, you can pin it in your browser and you can even just click it to get the same interface.
Our mnemonic phrase that secret phrase that we got has given us access to a new account.
Etherscan
The address of our account, we can use a tool like etherscan.
So if we look at this address that we just created, we can see that no transactions have happened.It's empty.It has zero ether.It has zero dollars worth of value.Address above is the unique address which represents exclusively the single account that we just created.
Multiple Accounts
We can even click the circle and create more account and give it a different account name.
Account 2 will have different account address.
Mnemonic , Public & Private keys
The mnemonic that we've been given gives us access to create multiple accounts with the same mnemonic.So havin that mnemonic will give us access to every single account that's ever created with that mnemonic.This is why securing your mnemonic is so crucial especially if you create multiple different accounts.Each account has a unique identifier with them as well.The account address is the public address.However there's also a private key to work with this account a secret key.We can view it by clicking three dots, go to account details and export private key.
This is a single password associated with this account.If you give somebody access to this private key, they'll have access to account 2.They won't have access to my account 1 because private key of account 2 is only associated with account 2.The mnemonic however is associated with all accounts.
Mnemonic vs Private vs Public keys
This is why when people say store your keys in safe place, they're referring to both private keys and mnemonic.If you lose your private key, you'll lose access to that account.If you lose your mnemonic you loose access to all your accounts.
Mainnet & Testnets
What else is going on in metamask is we can see this section here that says Ethereum mainnet.If we click it we can actually see a bunch of other networks in here.
When you buy eth and work with eth you're working on the ethereum mainnet.When you interact with smart contracts or defi you're going to be working on mainnet with real value.However since we're engineers oftentimes we're going to want to test our applications or do some type of integration tests or just make sure our code actually works.So there's also test nets.These are networks that resemble ethereum and work exactly the same way as ethereum does however they're not with real money and it's just for testing applications.We can even go to Rinkeby Etherscan and see the Rinkeby Testnet Explorer, we lookup our address and it's the exact the same information here.
Later on we're actually going to show you how to work with other evm compatible chains.Don't worry about evm compatible means for now but we can work with avalanche, polygon and other applications through networks interface as well.
Initiating our first Transaction
Infact we're going to make our first transaction on the Rinkeby Testnet and this will simulate exactly what it's like to make the transaction on the ethereum mainnet.
We're going to go to this application called the Rinkeby Faucet.This is where we're going to make our first transaction.Rinkeby is going to be one of the two testnets that we're actually gonna work with.The other test net that we're gonna work with is going to be Coven.It's important to know how to switch between testnets and evm compatible chains which is why we're going to be working with both.For now we're just be working with Rinkeby.
Faucet is a testnet application that gives us free test ethereum.Hence why it has no value because anybody can get it for free from one of these faucets.So to get testnet ethereum with faucet.rinkeby.io, we actually have to post a tweet or a facebook post with the tweet.
Now after the tweet we're gonna copy link to tweet and place it in Rinkeby Faucet and hit give me ether(18.75 ether/3days).
If we take the account address and go over to Rinkeby Etherscan, we now see the balance is 18.75.
But if we look at Ethereum Mainnet we've nothing there.We've just made our first transaction.And if we refresh the page, we can also see this is our first transaction that was made.Some account sent us 18.75 ether.We can even look at the details of the transaction.
Transaction details
Etherscan is what's known as Block Explorer.Block Explorer is an application that allows us to to see details of things that happen on a blockchain easily.We can see the transaction details of the transaction that we made and whenever we work with Smart Contract we'll also see them in a transaction similar to what we'll see in a transaction detail.
We can see a number of information above.We see a unique transaction hash.This hash or unique identifier uniquely identifies this transaction as the key of what this transaction is.
We can see the block number which we can see in a little bit.
We can see from this unique account which is the faucet account.
Gas fees, Transaction fees, Gas limit, Gas price
Gas refers to the fee paid to node operators for successfully including a transaction in a blockchain.Now exactly how this works is actually going to change pretty soon but the concept is basically anytime you want to change the state of blockchain wheteher this is a sending some ethereum or making any type of transaction, you actually have to pay a little bit of ethereum or a little bit of that native blockchain token to actually execute that transaction.Whenever we do something on the blockchain,it costs gas and if we do something that would take a lot of energy for the blockchain to do, it will cost more gas.So if I send single transaction that's gonna cost 21000 gas.However if I were to send a transaction that called the smart contract function and did a whole bunch of other stuff, it'd cost more gas.We see here whoever sent us eth also paid the blockchain miners or blockchain validators a little bit of ethereum to include our transaction.We actually get to pick how much of a fee we want to send with our transactions.
Gas vs Gas price vs Gas Limit vs Transaction fee
Why would we ever wanna pay more gas price? Why do I even have the option to pay more?
Blockchain can only process so many transactions at a time and nodes and blockchain nodes can only process so many at a time.So when I make a transaction a node has to decide why they want to include my transaction into the block.If there're ton of people looking to make these transactions then the nodes are going to be highly incentivized to pick the transactions that're going to give them a high price that are going to give them alot of money for including that transaction.
Gas estimator
In ethgasstation.info, we could get the gas estimator of the blockchain.
It currently says that if you wanna get your transaction in right away, it's gonna cost you 81 wei to do so.If you wanna get it in less than 5mins, it's gonna take take you about 62wei.
So the gas price of the ethereum fluctuate with how much people use it and the gas prices of all these blockchains fluctuate with how much people use it.
So typically when you're setting your gas price in a transaction, you can take a look-see at your gas station to see the price based on your needs but it all depends on how many people are looking to work with this blcokchain at the same time.
The reason that these gas prices exist is because nodes can only put so many transactions into a block so they're highly incentivized to input the transactions that are going to give them a higher fee.So in times when alot of people are looking to use a blockchain, prices will get very high and when very few people are using blockchain, prices will be very low.
With just this little bit of information, you now know how to interact with blockchains and interact with ethereum protocol.So if you don't want to learn how to code anything, you can go and you can start interacting with ethereum and interact with protocols with just this information.
Let's look under the hood of ethereum and what is actually going on with these transactions and with these gas and with these blockchains and what's really going on.Let's learn all the fundamentals of a blockchain.
** Hash or Hashing or SHA256**
Before we get started, we need to take a look at this thing that we call SHA256 hash.Hash looks like bunch of random numbers and it's a fingerprint of some digital data.
hash below starts with e3b0 which is the hash of nothing.
It just so happens it's a fingerprint of whatever I type in this box.If I type the text 'blockchain' into this box, you can see the hash has changed.
This is hash of the text 'blockchain' all lowercase which starts with 5318.If I delete the text and type the same text again, it'll have same hash.It's the digital fingerprint of the text 'blockchain'.
You could type tons and tons of text and you'd get a hash and the interesting thing about is if there's a tiny amount of information, no information or entire library of congress, you're always gonna get a hash that is this long shown above.You're not gonna be able to pre guess what it is.You'd always get exactly the same hash regardless of how many times you put exactly the same information in.
Block
Let's extend the idea of hash into something that we're gonna call block.
This is a block and it's exactly like the hash.It's just data section have been broken out into three sections: one called block:this is just some kind of number/block number, a nonce which is just another number and just some more data very similar to the way that we had it before. However the hash this block which includes all of the information up here in above figure which begins with four zeros.It's an relatively unsual hash and most of them are not really going to start with four zeros like the above hash.This one happens to and because it does totally arbitrarily, we can say the block is signed.
So what would happen if I were to change any one information in the above block ?
The above hash doesn't start with four zeros and the big background has turned red.Now we know this block with this informaton in it is not a valid or a signed block and that's where Nonce comes in.This Nonce is just a number that you can set to try to find a number that fits so that the above hash starts with four zeros again.
Above you can see mine button.When I click mine button, it's gonna run through all the numbers from 1 all the way up to try find one where the hash starts with four zeros and this process is called mining.
It stopped at 64,658 and that one just happens to hash out to something that starts with four zeros and it satisfies little definition of what a signed block is.
Blockchain
Blockchain is just the chain of these block.How do you put them together? Let's do that.
Block number 1 has nonce just like before, there's some data area too but then it has this Prev which is a bunch of zeros in first block.
The prev is block 2 starts with '0000ae' which is the hash of previous block (block 4).Each block points backwards to the one before it.You remember that the first block, there actually is no previous so it's prev is just a bunch of zeros.It actually just a fake number.
Just like we did before what happens if I change some information in block 5?
It's going to change the hash of this block and it's gonna invalidate it.
What would happen if I change something in block 4?
It's going to change the hash but that hash will get copied to the prev hash section of block 5 which change block 5 too.So it should break both blocks.
We can go back as far as we want to some point in the past and break that block and it'll break all the blocks since then.Everything before is green.
If I wanted to change something in the blockchian, I could go over to block number 5 and could change it then we could remine it and pick a different nonce.We could essentially alter the change.
What if I go back in past and break block 3?
Now I need to mine block 3 which picks nonce that makes the hash out to four zeros but block 4 and block 5 will still be broken.Although block 3 start with four zeros, adding the four zeros with different stuff in prev section of block 4, still makes the block 4 hash out incorrectly.So I also need to mine block 4 and similarly block 5.
What I wanted to tell is that if I go and change the last block, all I've to do is remine that last block.If I go way back in time to one of the block and make a change, I need to mine from that block.So more block I go by in the past, it's harder and harder to make a change.That's how a blockchain is going to resist mutation/change.
How do I know that my blockchain has been remined ?
Decentralized/Distributed Blockchain
Let's take a look at Distributed Blockchain.
It looks exactly like the last blockchain but has different peer for the blockchain.You can see peer B having the same blockchain structure like in Peer A.List could go out forever.There's many many peers out on the internet and they all have a complete copy of the blockchain.
If I look at the hash of Peer A block 5, it starts with '0000e4' and If I go down to PeerB block 5, it also has hash that start with '0000e4'.They must be identical.
If I type something on Peer B block 4 and remine block 4 and block 5.
All the block in Peer B is green.However the Peer A chain says that last hash start with '0000e4', Peer C says '0000e4' but the middle one says '00004c'.So just by glancing Peer B hash, something is wrong with that Peer B even though all of the hash start with four zeros.Essentially it's two against one.Peer A argues it's '0000e4', Peer B argues it's '00004c' and Peer C argues it's '0000e4'.So '0000e4' wins.So that's how a completely distributed copy having a copy on many different computers they can all very quickly see if all of the blocks are identical.Blockchains can have 400000/500000 blocks very easily so rather than checking through all of them, all you really have to do is to look at the hash of the most recent one and you can that if anything in the past was altered.You can tell by looking at the last block in the chain.
Tokens/Transaction History
That's the entire thing.There's no more to it than that but it's kind of not really useful because we don't have something in this data area that means anything I keep typing any text that's sort of irrelevant information.So what we really want is a token.Let's do a token on our blockchain.
Above tokens are totally arbitrarily which tells 25 dollars from 'Darcy' to 'Bingle'. 4.27 from 'Elizabeth' to 'Jane'.It's basically all the transactions that are happening.Replaced the data with these transactions and just like we saw before there's multiple blocks.It doesn't matter how many transaction there are.If we go down to other peers, we notice we have all these other copies of the same block.Now here's where the immutability is important.If I change something in Peer A block 4, the hash of the last block will be totally different from other peer's last block hash.If I were to go back in time and change some value, we would notice.That's the whole point of using the blockchain to resist any kind of modifications that have happened in the past.
We're not listing 'Darcy' has 100 dollars and she's 25 dollars to 'Bingle'.The only thing we're saying is 'Darcy' gives 25 to 'Bingle'.We're not remembering a bank account balance, only remembering money movements.So this begs the question "Does Darcy have 25$ ?"
We've a problem in this version of the blockchain: we don't actually know if Darcy has 25 dollars.So let's look at a Coinbase transaction.
Coinbase is gonna add a coinbase transaction to our blocks.This is saying that we're going to invest a 100 dollars out of thin air and give it to 'Anders' and there's no transaction in the block because nobody had any money previous to this.
In the next block, another 100 dollars comes out of nowhere and goes to 'Anders'.Now we've transaction and they're all from 'Anders' because 'Anders' is the only one who has any money at the point.'Anders' is sending 10 of his dollars to 'Sophie'.Do 'Anders' have 10 dollars? Yes he does because coinbase transaction has given him the 100 dollars.Add all the transaction 'Anders' is sending and they don't go over 100 and it follows sort of a basic rule of a currency that you can't invent it out of thin air, you can't create money out of thin air.It's dispersion is controlled.
If we look at this blockchain that we've created and zip forward in time and we notice that 'Jackson' is giving 'Alexa' 2 dollars.Does Jackson have 2 dollars? We go back a block before and see that 'Emily' had gotten 10 dollars from 'Andres' gave 10 to 'Jackson'.So 'Jackson' does have the money.That's actually one of the benefit's of having a previous in the block.It's easy to go backwards.
Summary
- Ethereum actually runs on Keccak256.
- Hash is a unique fixed length string to identify a piece of data.
- Hash algorithm is a function that computes data into a unique hash.
- Mining is the process of finding a "solution" to the blockchain "problem".
- Nodes get paid for mining blocks.
- Block is a list of transactions mined together.
- Blockchains are decentralized and distributed because many independent users are going to run this blockchain software and they'll check and compare against eachother to see which blockchains are acting honestly and which ones are acting maliciously.
- Nonce is a "number used once" to find the "solution" to the blockchain problem.
- Nonce is also used to define the transaction number for an account/address.
We're going to talk about signing the transactions, private keys and some other cryptography bacause in the blockchain below we see fantastc transactions but how do we know that it was Darcy who was the one to send 25 dollars to Bingley.How do we know that actually happened?This is where all those pieces that we just learned about in our test net in our metamask account are really going to come to life .
Public & Private Keys
Here we've an example of public and private keys.
Private key: Only Known to the Key holder, it's used to "sign" transactions.
You really wanna keep secret because you're going to be using this as a kind of your secret password for all the transactions .
I can really pick any private key I want with this Elliptic Curve Digital Signature Algorithm for ethereum and bitcoin they both use this elliptic curve digital signature algorithm.It's a variant of the digital signature algorithm and It's going to create it's public key.
That public key we want everybody to have access to right.Whole world can see this public key.
We're going to use the private key as like a password to quote/unquote digitally signed transaction and then people can verify them with this public key.
Let's see what this actually look likes
Signatures
If we go to signatures tab and let's say we have this message that we want let's say "hi world"
What's going to happen is the private key that we created, we can use to sign this data(hi world).We're using digital signature algorithm to create this message signature.
What's really powerful about how this algorithm works is that you can create this message signature with your private key but somebody else can't derive your private key from the message signature.This is what makes this really really powerful.
However if we go to verify using our public key, anybody can verify that the signature is yours.
Transactions
If somebody try to fake a transaction, they need to verify their's signature against my public key and very easily it turns red cause it's not verified.
We can take that into transactions into exact same way.
If I wanna send 20 dollars from my account to another account using my private key, I can sign that transactions and anybody else in the world could verify the transactions.This is why people say to protect your keys.
Your Ethereum address is actually a piece of your public key.To get our address to ethereum all we have to do is take this public key we've created with our private key, hash it using that same ethereum hashing algorithm and take the last 20 bytes and that's how we'll actually derive to our address.
Knowing the exact methodology of how to get the address doesn't really matter because it could change blockchain to blockchain but just know that's essentially how kind of these addresses are derived.
Note : This isn't how we send the transaction.So this is just going to sign it, create a transaction for us to send.We'll learn later on how to send these transactions.
Now we know all the cryptography pieces and all the little nitty grittes of how the blockchain actually works and how our signatures work and how everything sticks together.Let's talk a little bit about how this works in actuality and what's really going on.
For alot of this each different blockchain has slightly different algorithms, metrics and criteria for doing alot of this stuff.So when we're talking about specific implementations keep in mind the exact algorithm might be little different but the concepts are all still gonna be exactly the same.Hashing and hash function is going to be same no matter where you look.A decentralized blockchain is going to be same no matter where you look.How it's actually implemented is a little bit different.
Traditionally when you run an application you know be it as a website or something that connects to some server, you're interacting with a centralized entity and unlike how we saw with the blockchain with multiple different peers.It's going to be run by a single centralized group.It still could be run on many different servers but all those servers are still going to be controlled by the same centralized group.Blockchain as we saw run on a network of independent nodes.When we saw peerA, peerB, peerc and more, those were different examples of different independent users running the blockchain technology on their node.
Node
Node here means single instance of a decentralized system.When I say single node,when I'm talking about a blockchain, talking about one of those peerA's peerB's and so on running that blockhain software.I'm talking about one server running this technology and again it's the network it's this combination of these nodes interacting with eachother that creates this entire blockchain.What makes these so potent too is that anybody can join the network and that's why they're decentralized.The barrier to entry is a little bit of hardware requirements.Anybody can join these networks and participate and that's what make it truly decentralized.In fact you can go to github right now and run your ethereum node in a few seconds.
Centralized entity vs Decentralized Blockchain
In the traditional world applications are run by centralized entities and if that entity goes down or is maliciously bribed or decides they wanna shut off they just can because they're the ones that control everything.
Blockchain by contrast don't have this problem.If one node or one entity that runs several nodes goes down since there're so many independent nodes running that it doesn't matter.The blockchain and the system will persist so long as there is atleast one node always running.Luckily for us most of the popular chains like bitcoin and ethereum have thousands and thousands of nodes.As we've showed in our demo if one nodes acts maliciously, all the other nodes will ignore that node and kick that out or even punish it in some systems because they can easily check everybody else's node and see this one is out of sync with the majority and yes majority rules when it comes to blockchain.
Transactions are Listed
Each blockchain keeps a full list of every transaction and interaction that's happened on that blockchain and we saw if a node tries to act maliciously then all their hashes are going to be way out of whack and they're not going to match anybody else.This gives the blockchain incredibly potent immutabilty trait where nothing can be changed or corrupted.
So in essence we can think of a blockchain as a decentralized database and with ethereum it has an extra additional feature where it also can do computation in a decentralized manner.
Consensus
When we went through that blockchain example and we did that mining feature this is what's known as proof of work.Proof of work and proof of stake fall under the umbrella of consensus amd consensus is a really important topic when it comes to blockchain.
Consensus is defines as the mechanism used to reach an agreement on the state or a single value on the blockchain especially in a decentralized system.I briefly alluded to this consensus mechanism in our blockchain example when I said if one changes something and the other two don't then majority will rule and kick that one out.This is part of that concensus mechanism.
Now very roughly a consensus protocol in a blockchain can be broken down into two pieces.
Chain Selection AlgorithmSybil Resistance mechanism
Proof of Work/Sybil resistance mechanism
That mining piece we're doing or the proof of work algorithm is what's known as a sybil resistance mechanism and this is what ethereum and bitcoin currently use.Proof of work is known as a sybil resistance mechanism because it defines a way to figure out who is the block author: which node is going to be the node who did the work to find that mine and be the author of the block so all the other nodes could verify that it's accurate.
Sybil Resistance is a blockchain's ability to defend against users creating a large number of pseudo-anonymous identities to gain a disproportionately advantageous influence over said system and in laymen's term it's basically a way for blockchain to defend aganst somebody making a fake nodes so that they can get more and more rewards.
There are two types of sybil resistance that we're gonna talk about here namely POW (Proof of Work) and POS (Proof of Stake).
In POW this is sybil resistant because a single node has to go through a very computationally expensive process called mining which we demonstrated earlier to figure out the answer to the blockchain's riddle of finding that correct nonce.
In POW this works because no matter how many pseudo-anonymous accounts you make each one still has to undergo this very computationally expensive activity of finding the answer to the proof-of-work problem which again in our demonstration it was finding a nonce with that first four zeros but each blockchain might change the riddle or change the problem to be little bit different.
Blocktime
In fact some of these blockchain make this riddle intentionally hard or intentionally easy to change what's called block time.The block time is how long it takes between blocks being published and it's proportional to how hard these algorithms are.So these problems can actually change depending on how long they want the block time to be.
If the system want the block time to be very very long they just make the problem very very hard and viceversa.We'll ralk about civil attacks in a little bit and how they can affect the system but with proof of work it's a verifiable way to figure out who the block author is and be civil resistant.
Chain selection rule
Now you need to combine this with the chain selection rule create this consensus.There are some consensus protocols that have more features but very very roughly there are two pieces that we're going to look at.The second piece is going to be chain selection rule.How do we know which blockchain is actually the real blockchain and the true blockchain?
Nakamoto consensus
On Bitcoin and Ethereum they both use a form of consensus called Nakamoto Consensus and this is a combination of POW and longest chain rule.The decentralized network decides that whichever blockchain has the longest chain or the most number of blocks on it is going to be the chain that they use.This makes lot of sense because every additional block that a chain is behind it's going to take more and more computation for it to come up.That's why we saw on our transaction we actually saw confirmations.
Block Confirmations
The number of confirmations is the number of additional blocks added on after our transaction went through in a block.So if we see confirmations is two it it means the block that our transactions was in has two blocks ahead of it in the longest chain.Now I do wanna point out that a lot of people use POW as a consensus protocol and I do wanna say that this is a little bit inaccurate but sometimes people use it interchangeably.POW is a piece of overall consensus protocol which in bitcoin and ethereum one's current case is nakamoto consensus.
Block rewards & transaction fees
This has recently changed as of EIP 1559.
POW also tells us where these transaction fees and these block rewards go to.Remember how we made this transaction we had to talk about gas and a transaction fee.So who's getting paid? Who's getting this transaction and this transaction fee is going to the miners or validators?
In a POW network they're called miners and in POS they're called validators.There are little bit different and we'll get into that when we talk about POS.
In this POW system all these nodes are competing against eachother to find the answer to the blockchain riddle.Remember in our example it has to find hash that has four zeros at the start and again depending on the blockchain implementation that riddle is going to be a little bit different.
But all the nodes are trying as many as possible to try to get this answer first why because the first node to figure out the answer to the blockchain rule is going to get that transaction fee.They're going to get paid from that.
When a node get's paid they actually gets paid in two different ways.One is going to be with a transaction fee and another piece is going to be the block reward.
- Transaction fees
Remember we talked about alternating the gas price that's the transaction fee that we're gonna pay to these blockchain nodes for including our transaction.The block reward is given to the nodes from the protocol from the blockchain itself.
- Block Reward
You've probably heard of the bitcoin halving before.The halving is referring to this block reward getting cut in half and it's supposed to be cut in half roughly every four years.This block reward increases the circulation amount of whatever cryptocurrency that is being rewarded.For example on ethereum the block reward is giving out ethereum and on bitcoin it's giving out bitcoin.
So these nodes are competing against eachother to be the first one to find this transaction to be the first one to find the answer to this problem so that they can be the ones to win both this block reward and your transaction fee.Some blockchain like bitcoin for example has a set time when they're no longer going to give out block rewards and the mminers or the nodes are only going to get paid from transaction fees.
Gas fees are paid by whoever initialized the transaction.When we got our funds from the faucet there were some server and somebody else was paying the transaction fee for us.However we send ether from one account to another our first account actually paid some transaction fee to send that ether.
In POS there's also a gas fee but it's paid out to validators instead of miners and we'll talk about that in a little bit.
Now let's talk about two types of attack that can happen in this blockchain worlds.
Sybil attack
The Sybil Attack is when a user creates a whole bunch of pseudo-anonymous accounts to try to influence a network.Now obviously on bitcoin and ethereum this is really really difficult because the user needs to do all these work in POW or have a ton of collateral in proof of stake which again we'll talk about in a little bit.
51% attack
As we saw as part of our consensus protocol these blockchains are going to agree that the longest chain is the one that they're going to go with so long as it matches up with 51% of the rest of the network.This means that if you have the longest chain and you've more than 51% of the rest of the network you can do whats called a fork in the network and bring in the network onto your now longest chain.
Sybil attacks obviously are when a single node or a single entity tries to affect the decentrality of the network by pretending to be multiple different people although they're just the same person or entity and like I said it's really difficult to do in POW and POS.
Now you can see that blockchains are very democratic.Whichever blockchain has the most buy-in and is the longest is the blockchain that the whole system is going to corroborate.When nodes produce a new block and add to the longest chain the other nodes will follow this longest chain that the rest of the network is agreeing with add those blocks to their chain and follow up.So very small reorganizations are actually pretty common when a blockchain picks a block from a different longest chain puts it on and then has to swap it out for another block and continue with the different blockchain.
However if a group of nodes had enough nodes or enough power they could essentially be 51% of the network and influence the network in whatever direction they wanted.This is what known as 51% attack.This happened on blockchains like Ethereum Classic which is not Ethereum.This is why the bigger a blockchain is the more decentralized and the more secure it becomes.
Drawbacks of pow
POW is fantastic because it allows us to very easily protect against these sybil attacks and keep our blockchains decentralized and secure.However it has some drawbacks as well.POW costs a lot of electricity because every single node is running aa fast as they can to win this race to get the rewards.This leads to obviously an environmental impact.Since POW and Nakamoto Consensus a lot of other protocols have taken this idea and gone in a different direction with a different sybil resistance protocol.Alot of them with the intention to be alot more environmentally friendly and the most popular one rn is POS.
proof of stake/sybil resistance mechanism
There are some chains that are already using this Proof of Stake protocol and that are live and thriving.Some of them are like avalanche, solana, polygon, polkadot and terra and additionally ethereum has decided to upgrade to eth2 which will have this proof of stake algorithm as well.
It's a different sybil resistant mechanism instead of solving this difficult problem POS nodes put up some collateral that they're going to behave honestly aka they stake.In the example of ethereum 2 nodes put up some ethereum as a stake they're going to behave honestly in the network.
If they misbehave to the network they're going to be slashed or removed some of their stake.Obviously this is a very good sybil resistance mechanism because if you try to create a whole bunch of anonymous accounts then each one of those accounts you have to put up some stake and if you misbehave you're going to run the risk of loosing all the money that you put up as collateral.
Validators
In POS miners are actually called validators because they're no longer mining anything.They're actually just validating other nodes.Unlike POW which every node is racing to be the first one to find the block in POS nodes are randomly chosen to propose the new block and then the rest of the validators will validate if that node has porposed the block honestly.
As you saw with our cryptography lesson it's usually very easy for other nodes to verify if a proposal or a transaction is honest.
It's a really important topic when we're talking about blockchains because keep in mind these blockchains are deterministics systems.They're walled garded from the rest of the world and as you know a deterministic system by definition can't have random numbers.So how do we choose the random validators in the system?
Well it changes from blockchain to blockchain and actually choosing the node will change blockchain to blockchain but in eth2 they're using what's called randow at least for the original implementation.This is a decentralized autonomous organization that collectively chooses the random number and collectively chooses which node is going to run next.
pros & cons of pos
POS obviously has some pros and cons as well.
Pros are that again it's a great sybil resistance mechanism and a great way to figure out who the author of a block should be and the other pros are that it's way less computationally expensive to figure out new block because instead of every single node on the network trying to do this only one node needs to do this and the rest of the nodes just need to be validate.
Cons are that it's usually considered a slightly less descentralized network due to the upfront staking costs, it costs to participate.This gets into a little bit of a philosophical battle on how decentralized is decentralized enough and I think it's upto the community to decide and as we progress I think we'll learn more and more about how decentralized is decentralized enough.The general consensus among blockchain engineers though is that POS is very very decentralized and very secure.This massive environmental impact improvement is one of the two main reasons why eth is shifting to eth2.It reduces the environment impact by upto 99%.
Scalability problem & Sharding solution
- Scalability
When we're talking about gas prices we're saying that the gas prices can get really high if a lot of people want to send a transaction because a block only has so much block space and the nodes can only add so many transactions.So when alot of people wanna use blockchain the gas price skyrockets.This is not very scalable because we wanna add more and more people to these blockchains it's going to cost more and more to use the blockchains because more people are going to want to get into blocks.This means that there's kind of a ceiling to how many people can use the system because of the financial constraints will get imposed as gas prices keep rising. Eth2 is not only attacking the environmental impact of POW by switching to POS but they're also implementing this new methodology called Sharding.
- Sharding
Sharding is a solution to this scalability problem.A sharding blockchain really just means that it's going to be a blockchain of blockchains.There's a main chain that's going to coordinate everything amongst several chains that hook into this main chain.This means that there's more chains for people to make transactions on effectively increasing the amount of blockspace that there is.Sharding can greatly increase the number of transactions on a blockchain layer1.There's another term that might be the first time you've heard it's a layer1.
Layer 1 & Layer 2
Layerr1 refers to any base layer blockchain implementation.Bitcoin's,Ethereum and Avalanche a layer1.These are base layer blockchain solutions.
A layer2 is any application that is added on top of layer1;added on top of blockchain.Some examples of layer2 are going to be chainlink, arbitrum or optimism.Arbitrum and optimism are very interesting because they're layer2s that also look to solve the scalability issue.They are known as Rollups.
Rollups
Arbitrum and Optimism roll up their transaction into a layer1 like ethereum.We're not gonna go to deep into rollups and how they actually work but all you really need to know is that a rollup is kind of like a shraded chain.They derive their security from the base layer from the layer1 like ethereum and they bulk send their transactions onto the layer1.They solve some of the scalability issues by being another blockchain that people can make transactions on still on kind of base ethereum layer.Now they're different from side chains because side chains derive their security from their own protocols.Rollups derive their security from the base layers.So arbitrum and optisium for example is going to be just about as secure as ethereum.
Summary
- BTC and ETH are both Proof-of-Work blockchains that follow Nakamoto Consensus.
- ETH2 will be a Proof-of-Stake shraded blockchain.
- Sybil attacks are prevented due to protocols like POW & POS.
- 51% attack grows increasingly harder with the size of blockchain.
- Consensus is how blockchains decide what the state of the chain is.
- Sharding and rollups are solution to scalability issues on layer1's.
- A layer1 is any base blockchain implementation like bitcoin or ethereum.
- A blockchain scalability problem is that there's not enough block space for the amount of transactions that want to get in them which leads to very high gas price.
- Gas prices are how much it costs to interact with a blockchain.
Welcome to the remix ide.
This is where we start to work with solidity, smart contracts and deploy it to blockchains.
There's a bunch of solidity plugins like solidity, learneth, solhinthinter and more.But we're going to go ahead with solidity.We're going to create a little application that can store information on the blockchain for us.So we're going to create new file (.sol)
Solidity's compiler tab (Eth looking like on the tab) which compiles all the solidity code down to machine understandable code or machine language.Here's whole bunch of parameters we can choose when working with solidity like compiler version, choose the language, evm version.So let's code our first solidity contract.
We're going to use something a little bit special here when we deploy the code. We're going to use JavaScript VM which simulates actually deploying to a test net or a real network.We're not actually going to deploy on a real network.We'll in a little bit but just to get started we're going to work with JavaScript VM which is kind of a fake environment for now.
Testing locally and understanding how to test locally will make your coding experience a lot faster as you saw when we sent some transactions, some of them actually took a lot of time to actually deploy.We don't want to have to spend that much time waiting around for our tests to actually finish.
Solidity version
The first thing that you gonna need in any solidity program is the solidity version.That's always going to be at the top of your solidity code.It is defined by:
Ctrl + s will save and "compile".
If we want a specific version of solidity, we could also do:
But for this if we just hit ctrl + s it'll try to compile on previous compiler version.This might give error.You need to click the "Compile" button for compiler automatically switch to the specific compiler version.
We can also do:
pragma solidity ^0.6.0;
which allows us to work with any version of 0.6.
We'll be working with compiler version 0.6.0.However in future contracts that we work with we're actually going to work with different versions of solidity.The reason we're going to be changing versions throught the journey is that solidity is constantly updating language,being good at switching between versions is going to make you an incredibly powerful smart contract engineer.
Defining a Contract
To define our contract:
contract is a keyword in solidity.You can think of contract similar to a class in Java or any other OOP language.Here "SimpleStorage" is the name of the contract.
You could hypothetically deploy this right now and this would be a valid contract.
Variable types & Declaration
In solidity there're many different types that we can work with.Let's go into some of the types of solidity we can have.
You can also do:
uint x = 5;
But if you want to be little more precise, alot of protocols and smart contracts will do whole name like uint256.
We can also do bytes1, bytes2 upto bytes32.
Solidity Documentation
If you wanna learn more about the different types and the different variables that you can use, head over to the solidity documentation.
Initializing
Even if I didn't manually inilialize to 5, it'll get initialize to a null value.For now let's just not initialized it to anything that way it'll get automatically initialized to zero.This means when we deploy this contract as of right now, x will start off as zero.
Functions or methods
Functions or methods are self contained modules that will execute some task for us and in solidity it's the exact same thing they're defined by:
This is in simplest form of how you can define a function.This functions set's whatever value(whole number) we pass to the age.
Deploying a Contract
So let's go ahead and actually deploy this contract to actually interact with it.
If we hit eth looking button, which will bring us to the deploy tab and will allow us to deploy our smart contract.
Using our JavaScript VM it's given us fake accout with some ethereum in it.It has a 100 ethereum in it to start and same as before anytime we want to interact with the blockchain we have to pay a little bit of gas even in our fake virtual machine here.We want to simulate that so you'll see it has some of the same parameters here as making a transaction like gas limit for example.
When we deploy a contract, it's going to cost a little bit of ethereum or a little bit of gas to do so.Let's go ahead and click deploy button.
We can look at all the transactions we've recorded.We can see it says "Deployed Contracts" and we've our contract.
In this SimpleStorage Contract we see this big store button because there's one public function that we can actually interact with.So we can add 25 there, we hit store and look at our contract that we've paid little bit more gas to interact with the function.
Anytime we want to make a state change in the blockchain we've to pay a little bit of gas.The reason metamask isn't popping up is because we're kind of doing it in this simulated environment.
Public , Internal , private , External Visibility
This is great however it looks like we can't actually see what out age is.We can't actually look at it.So how do we actually make sure that we can view the age?
If we add public to our age, we recompile, delete our previous contract and redeploy and we can see button's pop up.
Let's talk about why this public variable allowed us to see this new button this age button?
This public keyword defines the visibility of the variable or the function.There are currently four different types of visibility.
- External: External function means it can't be called by the same contract.It has to be called by an external contract.
- Public: Public functions can be called by anybody including variables.Oddly enough variables are a function call to just look at them and return whatever that variable is.
- Internal: Internal functions can only be called by other functions inside of it's existing contract or in it's derived contract.
- Private: Private is the most restricitive as private functions and state variables are only visible for contracts they are defined in and not derived contracts.
The reason that we didn't see age in our original contract deployment is that if we don't give a state variable a visibility, it'll automatically get set to "internal".
Modifying a Variable
If we hit age button, it shows the value of age is 0.The store function is set so that whatever value(whole number) we pass it, it's going to change age to the value we pass it as.
If we pass 5, hit store, that transaction goes through and then the age we can see the value is now 5.
I'll be using transactions, smart contract interactions and function calls a little bit interchangeably.That's because on a blockchain whenever you call a function or whatever you make some state change to the blockchain you're actually also making a transaction.That's what makes this whole thing powerful and again that's why making a function call or deploying a contract costs a little bit of gas.
Scope
The reason we can access this age variable inside store function is because age has this global or contract scope.If we make a variable inside store function, we'd not be able to access it outside of a store function because it's self-contained inside the brackets{}.
View functions
You can also make a function called retrieve and make it a public function that is of type view and returns uint256.All this is going to do is return favorite number.
Question: "Why store function has orange color and other two has blue? "
The key relies in view function or view keyword.There are two special keyword that define functions that you actually don't have to make a transaction on.Those keywords are view and pure.
View functions
A view function means that we want to read some state off the blockchain.We're just reading off the blockchain.If we're reading off the blockchain and we're not actually making state change then we don't need to make a transaction.These blue buttons are blue because they're view functions.Public variables also have view functions that's why both of these are blue.Age variable is technically a view function.When I click it, I get to view the state of the variable.
Pure functions
A pure functions are functions that purely do some type of math.
We're doing some type of math above but we're not actually saving state anywhere.It's also going to have blue color button.
So for now let's remove public keyword from age variable and work only with retrieve function.
Structs
This application is great so far for it allows a single person to store a age and then retrieve it.But if we want a list of people or a group of people and store their age or what if we want to associate a age with a single person?
We've a whole number of different choices but the one we're going to talk about is using a struct.
Struct are ways to define new types in solidity.They're almost like creating new objects.
We've a new type of people that has a favorite number and a name inside of it.Now what we could do with struct is :
Let's deploy this contract and see what it looks like:
We've Humans struct which at 0th index is the age and 1st index is the name.
Intro to Storage
Storing variables in solidity always works in above numeric index fashion.In fact contract simple storage uint256 age is at index 0.
Arrays
Instead of just creating one person, we actually want to create a whole list of humans(boys in this case).So how do we create list of people?
Array is a way of storing list or a group of some object.
Dynamic Array
The array we created above is a dynamic array.It's a dynamic array cause it change it's size.Right now it's of size 0 and if we add something to it, it's of size one.
Fixed Array
This array could only have a maximum of one boy inside of it.
We're going to work with dynamic array because we want to add an arbitrary number of boys into.
Adding to an array
Let's go and create a new function called add boy.
We can see our new function addBoys where we pass a string memory name and a uint256 age.Then we create a Humans object and we push it onto our boys array.
Memory
In solidity there's more or less two ways to store information.You can store it in memory or in storage.
When you store an object in memory, it actually means that it'll only be stored during execution of the function or of the contract call.
Storage
If we hold it in storage, that data will persist even after the function executes.String in solidity is actually technically not a value type.A string is actually an arrayof bytes.A variables of type string is actually a special type of array that we can append.
Because it's technically an object we've to decide where we want to store in memory or in storage.Since we only need name during execution, we can have it be string memory name.Then when we create this new Humans object, we'll create a new copy of 'x' variable into storage.
Memory means after execution delete this variable and storage means keep it forever.
Let's deploy and see what happens now:
We've this addBoys function and since we're making a state change, here we can see that this indeed is a orange button instead of being a blue button.
Mappings
What if I'm looking for a boy's age from his name in the array?
Mapping takes some type of key and spits out whatever variable it's mapped to. In our case we want to find the age of the boy using his name.
Let's deploy and see:
SPDX License
Typically at the top of the contracts you want to add an SPDX license identifier.Basically solidity and ethereum community found out that trust in a smart contracts can be better established if source code is available and in terms of legality and copyright it just makes life alot easier if you add that license identifier right at the top of your solidity.
We're going to use MIT license identifier because it's the most open license out there.It means anybody can use this code and we don't care.
//SPDX-License-Identifier: MIT
Deploying to a live network
We now have a contract that we've decided that we liked.It got mappings, it enables us to actually store boys and their age.We've done all of our testing in this JavaScript VM and we've decided you know what we want to deploy this to an actual testnet or mainnet.
How do we actually deploy this so that other people can interact with this contract?
We're again going to use Rinkeby Test Network because that's what we use to make our first transaction.Now again you'll need some type of test ethereum in your test wallet.If you get lost you can always go to rinkeby faucet or a better alternative would be to go to the link token contracts in the chainlink documentation.
Only thing we need to change in remix is we need to change from JavaScript VM to Injected Web3 and metamask will actually pop up.
Injected Web3 means we're taking our metamask and injecting it into the source code of the browser.Web3 provider is if we want to use our own blockchain node or our own web3 provider.When we do injected web3 we're saying our web3 provider is our metamask will work perfectly for what we're trying to do.
If you deploy the contract into web3 provider, metamask will pop up asking if you want to do this transaction because remember we're deploying a contract we're changing the state of the blockchain,so we have to pay a little bit of gas fee for it.We get a link to the Rinkeby Etherscan similar to exactly what we saw before when we made a transaction.The difference here is that instead of sending ethereum to somebody, we're actually making a transaction on the blockchain to create a contract. After a short while it'll show up in status as success, the number of block confirmations which again is the number of blocks appended to block that included our transaction.
Interacting with Deployed Contracts
Same as working with the VM we've all the exact same functions in here.However if I click store button, metamask will pop up and do our transaction.
EVM
All the solidity code that we wrote and when we interacted with blockchain, all above solidity was compiled down to the EVM also known as Ethereum Virtual Machine.A lot of the blockchains out there today are what's called EVM compatible which means that all the solidity and all the functions that we're created can still compile down to EVM and deployed on their blockchain.
You'll find out a little bit later when we look to work on a non-ethereum based chain that we can still deploy our solidity smart contracts to these other chains as well.
Summary
- First thing you always gotta do in your smart contract is name the solidity version then name a contract.
- Contract in solidity is like a class and defines all the functions and parameters of your contract.
- There are many different types in solidity like unsigned integer, boolean, bytes, string and more.
- We can create structs, arrays, mappings, functions in solidity.
- View functions don't make a state change.
- Memory and storage are two different ways to initialize where a variable is going to be saved.
- All the solidity code that we work with gets compiled down to EVM.
We've got our first contract out of the way.We're understanding some of the basics of solidity.Now let's move onward into a little bit more advanced with our smart contract and let's build "Factory Pattern" of smart contracts.
Factory Pattern
We have our SimpleStorage contract which allows us to store age and age associated with different people. What if though I wanna have a lot these SimpleStorage contracts deployed? I wanna give poeple the ability to generate and deploy their own lists based off the SimpleStorage contract.
This is where a Factory Pattern comes into play.
New contract StorageFactory
Let's go ahead and create a new contract called StorageFactory.sol in the same location where SimpleStorage file lies.SimpleStorage and StorageFactory needs to be in the same folder.
Let's figure out how to get a contract to actually deploy another contract.We're gonna add those basics pieces that we added in that SimpleStorage contract.
Import 1 contract into another
So how can this contract deploy a SimpleStorage contract?
well first thing we're gonna do is actually import SimpleStorage into our StorageFactory contract.We need to import it so that our StorageFactory contract knows what a SimpleStorage contract even looks like.The way we can import it is:
Importing with that line is equivalent to copying everything in SimpleStorage.sol file and pasting it above in StorageFactory file.
It's interesting about having two contracts in the same file is that when you go to deploy, you'll actually have a choice of which one you want to deploy.
Deploy a Contract from a Contract
If we want this contract to be able to deploy a SimpleStorage contract we're of course have to create a function that can do that.
We're going to create an object of type SimpleStorage contract, name it "genSS" which is going to be a new SimpleStorage contract.SimpleStorage contract takes no input parameters.
If we deploy this contract as is by going to our deploy tab, choosing the StorageFactory and deploying.
We've the function that doesn't return anything.We're creating new contracts but can't really read where those contracts are being created.We'd have to look on a block explorer like Etherscan.
Track simple storage contracts
Let's make a way for us to keep track of all the different SimpleStorage contracts that we deploy.We put then in a list or an array.
We create a SimpleStorage contract array of visibility public and named it "SSArray".And Everytime we create SimpleStorage contracts, we'll add to our SSArray.
Let's see what happens when we deploy.
We've blue button which stands for our SimpleStorage contract array.If I click createSSContract, I've created a transaction that's going to create a new SimpleStorage contract and push it onto our SSArray.If I try to access 0th index, I'll get the address as shown in above picture.This is the address that the SimpleStorage contract was deployed to.
We've successfully deployed a contract to the blockchain from another contract.
Calling Store & Retrieve Functions from SF
We can do more than just deploying the contracts.We can deploy contracts from another contract and call those functions as well.
Let's create a function where we call store function and retrieve function of SimpleStorage contract from StorageFactory contract.
The reason I'm chosing a index is because we're going to choose which SimpleStorage contract in our list that we want to interact with.Then we're also gonna pass a age to call on the store function(it takes age as a parameter) of SimpleStorage contract.
Address & ABI
Anytime that you interact with a contract you need two things.
- You need the address of the contract you wanna interact with.
- ABI (Application Binary Interface)
for us we figured that out that we're going to push and get the address from the SSArray.We can get the ABI from the import command.We'll go into ABI later.
In order for us to interact with SimpleStorage contract, we can just do:
We created a SimpleStorage object (addSS) which stores the address of the contract that we wanna interact with.Then we pass the age by calling the method for that contract (same like in OOP) to store the age.
Let's see what happens when we deploy.
CreateSS contracts creates the contract and adds into our array "SSArray". sfStore which stores an age to one of the contract present in SSArray. SSArray shows the address of the contract.
**Adding Retrieve Function **
We can't see the age because we didn't add retrieve functionality.So let's add that.
index as a parameter is used to choose a contract on the SSArray and return it's age.Since we're only reading state so it's a view function.And we can return that object("addSS") retrieve method(like in OOP).
Compiling
If we compile above code and deploy SimpleFactory contract.We'll have sfRetrieve function which retrieves the age of the contract present in SSArray.
Making the Code lil bit Simpler
We can actually even refactor the code to be a little bit simpler.We don't need to save SimpleStorage contract address into it's object.We can directly call retrieve method and return that.Same goes for our sfStore too.
This is a way for us to actually deploy contracts and interact with contracts from another contract.
Additional Note
To deploy a contract we do need all the functionality of that contract imported.However to interact with the contract we don't need all of the functionality.We'll learn about interfaces in the next lesson which will allow us to actually interact with the contract without having all the functions defined.
Inheritance
SimpleStorage has got really cool functions and maybe I want all the functions inside StorageFactory.I want my StorageFactory to create SimpleStorage contracts and I want it to be a SimpleStorage contract itself.
Well my StorageFactory can actually inherit all the functions of SimpleStorage without me having to copy paste all the functions and variables over to StorageFactory.
I could do solidity's version of Inheritance.
Just by doing that line of code, my StorageFactory contract now will have all of the functions and variables of SimpleStorage.
Let's deploy the inherited contract.
Not only do we have all the functions originally defined in our StorageFactory but we aditionally have all the functions from the SimpleStorage.
Recap
- We learned to import entire chunks of code from other files into our files.
- We learned how to do inheritance.
- We learned how to deploy contracts from another contract.
- We learned to interact with different contracts from outside of our contract.
Same as before we're going to create a .sol file.We're gonna name it FundMe.sol and same as last time we add SPDX license identifier.For this lesson we'll choose solidity version >=0.6.6 <0.9.0.Create contract called "FundMe" and gonna start working.
purpose of this contract
We want this contract to be able to accept some type of payment.
Payable function , wei , gwei & ether
So let's create a new function that can accept payment.
When we define function as payable which means that function is used to pay for things.Every single function call has an associated value with it.Whenever you make a transaction, you can always append a value.That value is how much wei or gwei or finney or ether you're gonna send with your function call or your transaction.
As we mentioned before wei, gwei and ether are just different ways to talk about how much ether you're gonna send.If we look at ethereum converter, it shows how much gwei equals to 1 eth.
1 Wei is the smallest denomination of Ethereum.You can't break Ethereum into anything smaller than 1 wei.This is why when you're talking about how much something costs everything always defaults to wei.
Let's deploy the contract and see what it got.
The button is red because it's a payable function.If I click fund button, I can add value associated with it.
So what we want to do with this funding?What do we do when people send something?
Well Let's keep track of who sent us funding.We can create a new mapping between addresses and value.
Mapping , msg. sender , msg.value
In this fund function let's keep track of all the people who sent us money or all the addresses that sent us some value.To do this we can use some keywords that go along every transaction.
msg.sender and msg.value are keywords in every contract call and every transaction."msg.sender" is the sender of the function call and "msg.value" is how much they sent.
So whenever we call fund function, somebody can send some value cause it's a payable function.And we gonna save everything in this "addressAmountFunded" mapping.
Let's deploy the contract:
We've a new view function "addressFundedAmount".
Funding
If I click directly fund, nothing gonna happen because my address is going to be sending zero.In order for me to send something, I've to add some value along with my transaction.So above we've send 1 gwei which is equal to 1000000000 wei.Before hitting fund let's copy the fake account address to retrieve the funded amount.We've called a fund function with a value of 1 gwei associated with it.If I call the addressFundedAmount with the same address, I can see how much I've funded this smart contract.We can even add more value.
When we send funds to our contracts this actually means that this contract wherever this is deployed, now is the owner of value of addressFundedAmount of ether.We now have a way to fund smart contracts.
ETH -> USD /conversion
In this smart contract, in this funding that we're doing we want to create a minimum value for people to be able to fund our endeavors which whatever they may be.We want to set some minimum value. Ether is great but for whatever reason we wanna work in USD or maybe in some other currency.
How are we going to get the conversion rate from that currency to a currency that we can use in this smart contract?
First thing that we're gonna need to do to set the value is to know what the Eth to USD conversion rate is because If I want to accept ethereum as the token but I want it in it's USD currency then I'm gonna need to know what that conversion rate is.
Deterministic problem & Oracle solution
So How are we going to get this data into smart contract? Where are we going to get this data from?
Remember how we talked about blockchains being deterministic systems and oracles being a bridged between blockchains and the real world.Well this is exactly where oracle's come in.
When we're talking about these systems you know these blockchains they cannot connect to real world events.They can't connect to external systems.They can't do external computation.They're intentionally these deterministic systems.These walled gardens.In order for us to make the connection, we need a blockchain oracle.We need some type of network.
If you look at a blockchain, it can easily say 1 + 1 = 2 and every other node can easily verify this.However blockchain can't easily say let's all grab the same random number because each node is going to get a different random number.They also can't make an API call because if one node calls the API and at a different time another node calls it or specifically an http get, there could potentially get very very different results and if another node tries to replay these transactions by calling these API's again maybe 10 years in the future, there's a good chance that API is going to be depreciated and they could be hacked, malicious etc.
The other reason that blockchains are intentionally can't make an API calls because then they would be making assumptions about real world and layer1's typically don't want to have an opinion on any political or geopolitical issue.Whereas oracle network on the other hand can make those assumptions.
Centralized Oracles
The other main thing we need to talk about here is centralized oracles being main points of failures.If you or I say "I'm just going to be a the oracle.I'm going to be the one to put this data on chain.", we now have this massive centralized point of failure.We've done all this work to make our decentralized computation decentralized and on chain but we ruin all the decentrality by having a single point of failure.Remember one of the whole purposes of blockchain is so that not a single entity can flip a switch and restrict our freedom to interact.
With a centralized oracle a single entity can flip a switch and restrict our freedom to interact with eachother.
Decentralized Oracle Networks
We also need to get data from many different decentralized sources or do any type of computation in a decentralized manner.This is where chainlink really shines.
Chainlink is a modular decentralized oracle infrastructure and oracle network that allows us to get data and do external computation in a highly sybil resistant decentralized manner.
It can be as customizable as you want.As you can run with one node or many nodes or do as many nodes as you like.
Chainlink Datafeeds
Currently one of the most popular features of chainlink is their data feeds or their price feeds.We can even go check them out over here.
We can see number of different price feeds and the networks that are providing the prices for these specific pricing powers.We can see above there's a whole number of decentralized different oracles returning data for different price feeds.This above one is ETH/USD and it's exactly the price view we're looking for.
Having a decentralized network bring the data on chain and have it as a reference point of difinitive truth allows users to all collaborate and use this common good and it'll be cheaper, more secure and more efficient than anybody even running their own centralized oracle.These price feeds are incredibly powerful.
Additionally they're being used by some of the top protocols in the defi system right now like synthetix is securing around 2 Billion $, sushi swap for leveraging trades, set protocol commodity money, ave for understanding the price of an underlying collateral.This is an example of an out of box decentralized solutions that's already been packaged in the decentralized manner for you to consume and for you to use.This makes going to production a thousand times easier than building everything yourself
Chainlink Code documentation on ETH/USD
However if you want to make API calls and build your own decentralized network, you absolutely can with the chainlink API calls.We're not going to go into that here because using chainlink pricefeeds, chainlink vrf, keepernetwork and all these other pre-box decentralized services are going to make going live and going mainnet a lot easier.
You can always make a chainlink http get call as well.We're not gonna go over this because putting this into production is a little bit tricker.
If you ever want to try them out by themselves, you can head here and click "open in remix".It'll kick us to remix edition with all the code written for us.Let's go ahead and compile it and deploy it to real network(Injected Web3 in Environment).
The code is actually for 'Kovan network'.So switch your metamask to Koven.Grab Koven faucet if you don't have any.Let's deploy the contract, metamask is gonna pop up and we can get the latest price by clicking the button.
We can see the function does indeed returns the latest price of the ethereum.You might be asking "Why this number look so big?"Remember how we talked about wei, gwei and either.Well the reason that those exist is because decimals don't work in solidity.We actually have to return a value that's multiplied by 10 to some number.
If we get a value of 261497384316, this value is actually 2614.97384316 * 108.
Next question you might wanna ask is why did we work with this on a testnet? Why can't we do this on a local network? The answer to this is because there're no Chainlink nodes on a simulated JavaScript VMs.We'll learn later how to actually mock these interactions and mock a chainlink node returning data onto our blockchain.But for now let's head back over to the contract that we're working on so we can learn how to implement this latestprice in any contract that we ever want to.
LatestRoundData
Another contract called in this case called priceFeed has a function called latestRoundData which returns alot of data.
It returns a roundID which defines how many times this priceFeed has been updated, returns the price which is the actual conversion between two assets, returns a startedAt which defines when this was last updated, returns a timestamp and returns a answeredInRound.Don't worry about answerInRound for now.If you want to dive deeper into what these rounds mean and what answeredInRound means you can definately check out the chainlink documentation.
Importing Datafeed code from Chainlink NPM package
How do we implement this data feed into our fundMe application?
First thing we actually need to do is import the chainlink code.
As we know an import take whatever code you're importing and stick it at the top of your project.When we import from the "@chainlink/contracts", we're actually importing from the "@chainlink/contracts" npm package.We can see and read more about this repository here.
Interfaces
If you follow that import path that we got from documentation, we'll end up on the file below.
It's what we call an interface.These contracts don't start with the contract keyword but start with interface keyword.The main difference is that you can see their functions aren't completed.They just have the function name and it's return type.
In our code here solidity doesn't natively understand how to interact with another contract.We've to tell solidity what functions can be called on another contract.This is where interfaces are actually going to come in.Similar to structs what we can do with interfaces to find a new type.
ABI/Application Binary Interface
Remember how we said before we talked a little bit about ABI.Interfaces actually compile down to ABI.It tells solidity what functions can be called on another contract.We need solidity to know what functions it can use and what functions it can call other contracts with.
Anytime you're going to interact with another contract in solidity or smart contract programming in general, you're going to need that contracts ABI.We'll go into what these ABI's looks like a little bit later.
Interacting with an Interface Contract
How do we work with interface contract?
To interact with an interface contract it's going to work the exact same way is interacting with struct or a variable.
Let's define a new function called getVersion and we're going to call the version function of interface on our contract.
The same way we define variables and structs, we define working with other contracts and interfaces.
First thing we named is type which is AVI.Since we're inside of a contract, we're going to skip visibility and give the name "rate".Then we initialize a contract.How do we actually choose where to interact with the AVI contract?Well we pass the address of where the contract is located.
Finding the Pricefeed Address
In order to find where this ETH/USD pricefeed contract is located on the Rinkeby chain, we can look at the ethereum price feeds chainlink documentation.It has a ton of different price feeds and even more not price feeds related data.Scroll to Rinkeby because on each different chain the contract address that has all the price feed information is going to be different.Scroll down and find ETH/USD.
Copy that address and pass it to AVI initialization.
It's saying that we've a contract that has "AggregatorV3Interface" contract's function defined in the interface located at that address.If that's true, we should be able to call "rate.version".
Deploying
Let's compile it and deploy it to testnet(Injected Web3 as Environment).Remember that address is located on a actual testnet.On an actual network.We'll learn later on how we can actually work with a simulated chain and work with these price feeds but that's much later in the journey.
We can see that the version of our aggregatorV3Interface is version 4.So we just made a contract call to another contract from our contract using an interface.This is why interfaces are so powerful because they're a minimalistic view into another contract.
Getprice function
This is great we've a getVersion function but this still isn't the function that we want.We want to call a getprice function which if we look at our interface, we can see there's a latestRoundData function that returns an answer.
Let's go ahead and make a function that calls getprice instead.
Tuples
This latestRoundData function returns five variables.So how do we actually work with that?
A tuple is a list of objects of potentially different types whose number is a constant at compile-time.We can define several variables inside tuples.
Since lastRoundData function returns five different values, we can also make out contract return those five values.That above is a syntax for getting a tuple.Although our compiler will give us some warnings, it's saying unused variables because we're not using them for anything.We'll come back to that.
Then we can pick the variables that we want to return. Answer is the rate so we return that.
Typecasting
But compiling the code gives an error.It says return type argument int256 is not implicitly convertible to expected type."Answer" is an int256 and we want to return uint256.How do we rectify this?
We can fix the error by typecasting.Integers in solidity are really easy to cast into eachother.We could just do:
Now our compiler is happy.This getPrice function should return the latest price of Ethereum in terms of USD.
Deploying
Let's go ahead and deploy this contract.
We can know the rate of Ethereum in terms of USD is 2746.19432590 $.
Clearing unused Tuple Variables & Deploying
Let's clean up the function before we go up.As you can see one thing the compiler is complaining about is we've unused local variables but latesRoundData returns five different variables.How do we actually return the five variables but make our compiler happy with us?
We can actually return blanks for each one of the unused sections with commas in between eachother like:
Wei/Gwei Standard (Matching Units)
Let's put everything into the gwei/wei standard.We saw that getPrice has 8 decimal places.However the smallest unit of measure, it has 18.So typically let's make everything have 18 decimals as well.You don't have to do this and it'll save some gas if you don't.We could do:
return uint256(answer * 10000000000);
getting the price using Get conversion rate
We've the price of Ethereum in USD.We could set the price of our funding function to anything that we want.For example let's say 50$.We could convert whatever value they send us to it's USD equivalent and see if it's greater than or less than 50$.
Let's make a new function that converts that value that they send to its USD equivalent.
Let's test this out and see why we have to do one more thing?
This seems like a really big number i.e 2746194325900.00000000.This says that 1gwei is equal to 2746194325900 in USD.The price of 1ETH is not even that much.The reason that's off is we've to divide by 100000000.Both ethPrice and fundedAmount has 1019 tacked on to them.
Safemath & Integer Overflow
Since we're on the topic of math, let's talk briefly about some of the pitfalls of solidity especially when it comes to math.
Prior to solidity 0.8 if you added to the maximum size, a uint number could be wrap around the lowest number that it would be.For example:
if we add two uint8 number : 255 + uint8(1) = 0 255 + uint(100) = 99
This is because integer can actually wrap around once they reach their maximum cap.They basically reset.
This is something we need to watchout for when working with solidity.If we're doing multiplication on really big numbers, we can accidentally pass this cap.Luckily as a version 0.8 of solidity, it actually checks for overflow and it's defaults to check for overflow to increase readability of code even if that comes a slight increase of gas costs.
Just be aware if you're using a lower version that 0.8 you're going to have to do something to make up for this.
We could write whole bunch of code to check all of our math or we could just import "SafeMath" from another package.Similar to chainlink we can import SafeMath from tool called OpenZeppelin.
OpenZeppelin is a open source tool that allows us to use a lot of already pre-built contracts.
Libraries
Libraries is similar to contracts, but their purpose is that they are deployed only once at a specific address and their code is reused.
Using keyword: The directive using A for B; can be used to attach library functions (from the library A) to any type (B) in the context of a contract.
In this case we're attaching a SafeMath chainlink library to uint256 so that these overflows are automatically checked for.
This is for those of you who are familier with SafeMath and integer overflows and underflows.We're not going to be calling the functions that SafeMath provides us like div, add, mull all those functions.Simply because in 0.8 moving forward we no longer have to use those.We can just use regular operator like '+' & '-'.
Setting Threshold
We know have a way to get the conversion rate of whatever eth is sent and turn it into USD.Now we can set a threshold in terms of USD but how do we guarantee that whatever amount that the users send when they call fund is going to be atleast 50$.
First set the minimum value by:
Require statement
Now that we have a minimum amount how do we actually make sure that this minimum amount is met in the value they send us?
We could do that by:
When a function call reaches a require statement, it'll check the truthiness of whatever require you've asked.In our case the converted rate of msg.value needs to be greater than or equal to our minUSD.If they didn't send us enough ether then we're going to stop executing.
Revert If the converted rate of msg.value is less than 50$, we're going to stop executing.We're going to kick it out and revert the transactions.This means user gonna get their money back as well as any unspent gas and this is highly recommended.We can also add a revert error message.
Deplying & Transaction
Let's go and deploy the contract.If I try to fund less than 50$, below error message will be displayed.
The contract isn't even letting us to make the transaction.Whenever you see gas estimation failed errors usually that means something reverted or you didn't do something that was required.
Withdraw Function
Now we can fund this contract with a certain minimum USD value.You'll notice though that right now we don't do anything with this money.We're going to fund this contract however that's it and we don't have a function in here to actually withdraw the money.There's no way even though we just sent this contract some money.There's no way for us to get it back.How do we fix this?We could add a withDraw function.
This is also going to be a payable function because we're going to be transferring eth.
Transfer , Balance , This
Transfer is a function that we can call on any address to send eth from one address to another.In this case we're transferring ethereum to msg.sender.We're going to send all the money that's been funded.So to get all the money that's been funded, we did address(this).balance
this is a keyword in solidity.Whenever you refer to "this", we're about contract that you're currently in and when we add address of this we're saying we want the address of the contract that we're currently in.
Whenever we call an address and then the balance attribute, you can see the balance in ether of a contract.So with that line we're saying whoever called the withdraw function because whoever calls the function is going to be a msg.sender transfer them all of our money.
Deploying
Let's fund the transaction with lots of ether.We fund it with one whole ether, hit the fund button and we're sending 1 whole ether into this contract.If we look at our balance it's will get down by 1 ether.Let's try to get it back.If we call withdraw function, once the transaction goes through we should get all of our ether back.
Owner , Constructor Function
Maybe we don't want anybody to be able to withdraw all the funds in this contract.We want only the funding admin to be able to withdraw funds so how do we set this up in a way that only the contract owner can actually withdraw funds?
Well we learned before that the require function can actually stop contracts from executing unless some certain parameters are met.We can do the same thing here with:
require msg.sender = owner
But we don't have an owner to this contract yet.How do we get an owner to this contract the instant that we deploy it?
We could have a function called createOwner but what happens if somebody calls this function right after we deploy it then we wouldn't be the owner anymore.
So we need a function to get called the instant we deploy this smart contract and that's exactly what the constructer does.So typically at the top of your smart contracts, you'll see a constructor and this is a function that gets called the instant your contact gets deployed.
Deploying
Let's deploy the contract now.We can see our address as the owner of the contract.
After we've owner, we can go to withDraw function and set the require statement.
After we deploy again, if the contract has same address that deploy to withdraw, only then it'll successfully withdraw.
Modifiers
We can now require this withdraw function is only callable by the owner.Now What if we have a ton of contracts that want to use require(msg.sender == owner)?IS there an easier way to wrap our functions and some require or some other executable?
This is where modifiers come in.We can use modifiers to write in the definition of our function, add some parameter that allows it to only be called by our admin contract.
Modifiers are used to change the behaviour of a function in a declarative way.Let's create our first modifier:
What a modifier is going to do is before we run the function do the require statement first and then wherever your underscore is in the modifier run the rest of the code.
Now what we can do is make the withDraw function as admin.What's gonna happen is before we do the transfer, we're actually gonna check the modifier which runs the msg.sender == owner.
![withdrawadmin][/Images/Day5/e39.png]
Deploying
Let's deploy the contract in JavaScript VM, we can call withDraw from the address of the deployed account only.
Resetting the Funders Balances to Zero
The only thing that we're really missing is that when we withdraw from the contract, we're not updating our balances of people who funded this?So even after with we withdraw this is always gonna be the same.We need to go through all the funders in this mapping and reset their balances to zero but how do we actually do that?
For loop
We can actually loop through all the keys in a mapping.When a mapping is initialized, every single key is essentially initialized.We obviously can't go through every single possible key on the planet.However we can create another data structure called "Array".
Let's go and create a funders array that way we can loop through them and reset everyone's balance to zero.
Summary
- Right away when we deploy this we're set as owner.
- We can allow anybody to fund.
- They have to fund it with minimum USD value that we actually set.
- Whenever they fund we keep track of how much they're funding and who's been funding us.
- We can get the price of Ethereum that they send in the terms of USD.
- We can convert it and check to see if they're sending us the right amount.
- We have our admin modifier so that we're the only ones who can withdraw from the contract.
- When we do withdraw everything from the contract, we reset all the funders who have currently participated in our crowdsourcing application.
Deploying & Transaction
Let's see if everything works end to end.If we fund 0.1 eth to the contract, and we look at the 0th index of funders, we'll see the address that funded the contract.Even other account could fund the contract.
Forcing a Trasacttion
Let's try to be malicious.Let's have another account withdraw all the funds in here.If another account clicks withdraw function, the transaction will fail.We're relentlessly malicious we want to send the transaction regardless so even though I'm not the admin of the contract I've gone ahead and still tried to send those withdrawl.So what happens now?
We'll see remix saying something went wrong.
We've been working with remix so far to start our smart contract in our solidity development journey.Remix is an incredibly powerful web IDE .Remix should always be the starting ground for anybody looking to start their smart contract journey because it's a wonderfully friendly way to really show what's going on behind the scenes and it's easy to see everything we're doing with ethereum, chainlink and our smart contracts.
Limitations of Remix
- Can't integrate other parts of a project.
- Limited support for tests or custom deployments.
- Can't save files locally without a plugin.
- Any custom functionality requires a plugin.
- Needs an internet connection.
- Doesn't have python.
In order for us to deploy, test and automate everything about our smart contract development cycle, we want to connect our solidity and our smart contracts with more traditional programming language like python.This way we can customize our entire development environment in any way we like.
First we'll learn to work with web3.py which is an incredibly powerful python package for doing everything we wanna do with smart contracts.After we learn some basics of web3.py, we'll move on to brownie which is a smart contract development framework built on top of web3.py which makes our life even easier.However it's really important to learn web3.py because this will teach us what's going behind the scenes of brownie.
VScode , Python , Solidity Setup
For the rest of the journey I'm going to be working with visual studio code which is an incredibly powerful text editor that will give us a lot of formatting and a lot of really nice tools to work with deploying and interacting with our smart contracts.
You can download VScode here.
extensions to installed:
- python3
- solidity
VScode features
One of the amazing things about VScode is you can actually open a terminal up inside a visual studio code. You can go over to the top bar to terminal and select new terminal.
You can actually hit ctrl + (backtick) to toggle back and forth between having terminal open and closing it.
Testing python install & Troubleshooting
We can test in terminal to see if python is installed correctly.If we type python --version, you should get something like:
If it doesn't work you can look for troubleshoot in google.Whatever the error you've, you'll get a link which will lead you to the answer.
Creating a new folder
In our terminal we can create some folder.You can create one by doing:
mkdir folder_name
and to go inside the folder:
cd folder_name/
SimpleStorage. sol
We'll be working with SimpleStorage again the exact same contracts but instead we're going to be using web3.py.
We want our visual studio to know that we're in the folder where we wanna work.We can click files icon, click open folder and open the folder that you wanna work in.
Remember to save
Now the file is in VScode, we can see we've little dot here.
Whenever you see that dot, it means that your VScode file isn't saved.We wanna always save it otherwise when we compile, things might not work correctly.
We can see there's line under pragma.It's VScode way of telling there's an error at this position.This is just the extension being a little bit confused.We can safely ignore this but if it's really bothersome you can:
right click it and do something like solidity change global compiler version or
go to perferences then settings.Type solidity and change the remote compile version to 6.0.6
Also we wanna fave format on save as checked.This allows to format our code whenever we save the code.
Python Formatter & settings
First thing we're gonna do is installed a black python formatter.To install it type this command in terminal:
pip install black
search for python formatting and set formatting provider to 'black'.
Author's recommended Settings
It's highly recommended to have both format on save for both python and solidity.
working with python
Let's go ahead and create a new file and we'll call it deploy.py.
Reading our solidity file in python
First thing we wanna do is read SimpleStorage solidity file.We need to get the solidity file into deploy scripts so that our python file knows what it's going to deploy.
Keyboard Shortcuts
To look at keyboard shortcut's you could hit ctrl + p, add > type keybord shortcut references to see all the shortcuts reference.It'll bring to the keyboard reference page based off of what OS you've.
Py-Solc-x
Now that we can actually read from our SimpleStorage.sol file we actually have to compile it because back in remix every single time we did anything with our files, we'd to compile them first.We need some compiler in python.Luckily there's a fantastic python package called Py-Solc-xYou can install it by:
pip install py-solc-x
Importing solcx
The way we can use it is by importing it into our python file.
Compiled_sol
We're going to save our compiled code to compiled_sol variable.
We're not gonna go too deep into what's this settings and output selection are actually doing but if you want to learn more you can go to the homepage of pysolcx documenation.
You'll see we get the massive object which has whole bunch of basically unreadable pieces but that are the low level code that actually gets compiled whenever we use the compiler in remix or in python.
Remix actually does the exact same thing.Once we compile our contract, you can actually copy the byte code.You'll see whole bunch of stuff.There's also a opcode which are low level code that the contract are actually doing that actually governs how the code works.
You'll also see ABI which we're gonna output in our deploy.py.If you copy and past the ABI in new file, you'll see long json object.It explains all the functions and variables.It's a lowest digestible way to say here's where all the functions are,here's what the parameters types are, return type gonna be and everything like that.
Saving Compiled Code/writing
This takes our compiled_sol json variable and dump it into the compiled_code.json file.It's going to keep it in the json syntax.
Deploying in Python (Bytecode , ABI)
We've compiled our solidity and stored our solidity code to compiled_code.json file.Now we probably want to deploy it and test it out.So how do we actually do it?
First we actually get the byte code.We need the bytecode of the file so that we can actually deploy it.
Now that we have out byte code we also need ABI.
Now we've two main pieces to deploy, all we've to do is deploy.
Which Blockchain/Where to deploy
The question that becomes is where are we gonna deploy it to.Which blockchain are we going to deploy to?In remix when we're first playing around, we're using a JavaScript VM, we absolutely could and we'll learn to deploy to a testnet because that's going to be the same way that we're going to deploy to a mainnet.But before we do that we should learn how to deploy on a simulated environment.
Ganache Chain
This is where Ganache going to come to the rescue.Ganache is a simuated blockchain that we can actually use to deploy our smart contracts to and have it interact like it's a real blockchain.Ganache is going to allow us to spin up our own local blockchain.
You can download Ganache from here.
User interface is really nice because it allows us to do one click blockchain or create our own local blockchain.That means this blockchain isn't connected to any other blockchain out there but it'll act like a blockchain and will be lot faster than us having to interact with a testnet and we control the entire blockchain because it's only one node.Ganache is our JavaScript VM.
Ganache UI
We go ahead and click quickstart which will automatically upload and get started with our own local fake blockchain.
You can even see it gives us some accounts.Each one of these addresses has a private key.In your ganache if you click key, it'll show key.
Let's learn how to connect to this ganache blockchain from the UI first and then we'll learn how to do the command line version.
Introduction to Web3. py
This is when we finally start working with web3.py.we just do below command in VScode terminal:
pip install web3
Now we can start working with web3.py.
Http/Rpc provider
To connect the contract to blockchain we choose http provider.If we look at ganache instance, we've rpc server which has the url HTTP://127.0.0.1:7545.This is the url that we're going to use to connect to this blockchain.In remix we're actually using our metamasks directly to connect to the blockchain.However we wanna connect directly to our simulated blockchain.
Connecting to Ganache(RPC server,Documentation,Chain ID,address,Privatekey)
With everything that we show you, you probably going to want to get really familier with the documentation because even after being a pro, you're going to want to use it more and more and more if you want to learn about other providers.You can go to providers page of the documentation.
The next thing that we always gonna need is chain id or network id.What is the id of the blockchain? For ganache it's 5777.We also need address.We can grab fake address to work with.
Similar to how in remix when we working with the JavaScript VM we're given a bunch of fake addresses.We're doing the same thing but with ganache.Then we also want our private key.We need the private key to sign our transactions.
Whenever you add a private key in python, you need to add 0x to the front.Python is always gonna look for the hexadecimal version of the private key.
Now we've all the parameters we need for interacting with and connecting to our ganache local chain.
Deploy to Ganache
It's time to finally deploy our SimpleStorage.sol contract.Let's do it.So the credit contract that we're going to deploy with web3.py.
Does this mean we've deployed it ?Well no this just means we've a contract now.We've created a contract object.
How do we actually deploy this?
Building a Transaction
we need to actually build our transaction because again whenever we interact with the blockchain, whenever we make a state change and in our case we'd be deploying a contract we're going to make a state change So we need :
- Build the contract deploy transaction
- Sign the transaction
- Send the transaction
To do all that we need to talk about Nonce thing.
Nonce
Remember way back in our blockchain demo when we used a nonce to solve the answer to that really difficult mining problem.Well defination of nonce is just a word coined or used for just one occasion and in cryptography it's an arbitrary number that can be used just once in a cryptographic communication.The nonce that is used to find the answer is going to be different from another nonce that we're actually gonna need to make our transaction.
If we look at our metamask, look at our activity and look at one of the transactions we've made recently on etherscan, we can see nonce here as well.
This nonce is the number of transaction that our account has actually made.Everytime we make another transaction, our transaction is hashed with a new nonce.This is what going on behind the scenes with our transaction and we need this to send our transaction.
Getting Nonce
We can actually get our nonce by just grabbing our latest transaction count
We'll get our ans as 0 if we print the nonce because on our local blockchain the address we're using hasn't been used before.
Create a Transaction
Let's create a transaction object.
As you might have pointed out our SimpleStorage.sol doesn't actually have a constructor.Every contract technically has a constructor.In our case it's just blank.We're not telling our SimpleStorage.sol to do anything.
Transaction Parameters
In web3.py we need to give atleast a couple of parameters.We always have to give chainId, from and nonce.
If we print the transaction, we could see even more parameters.We have value which is the ether that we're going to send, gas, gasPrice which we can arbitrarily set if we'd like, chainId, from, nonce, data and to which is just empty because it's sending it to the blockchain.
The giant data object there is encompassing everything that's happening in SimpleStorage.sol.
Signing Our Transaction(signed_txn)
This is just a transaction and anybody could actually send the transaction as long as it's signed by them.So we've the transaction but we need to sign it from somebody.Since we're sending it from our address, our private key is going to be the only key that's going to work to sign the transaction.
Remember when we were talking about public private keys, we right now have a message that is defining how to deploy SimpleStorage but it's not signed yet.So we're gonna need to use our private key to sign it to create unique message signature that we're the only ones that can create the private key but anybody else can verify it was us who signed it.
If you encounter the following error:
- Transaction must not include unrecognized fields
- ValueError: Method eth_maxPriorityFeePerGas not supported
Never Hardcode your Private keys
It's a really bad practice if you push the code to source or github.Somebody else can see your private key and steal all your funds.So we don't wanna hard code our private keys in our code like we're doing here.
Let's take this time to talk about environment variables and how to set them.
Environment variables
Environment variables are variables that can set and then we set in our terminal and in our command lines.Following is a way to set an environment variables in MacOS and Linux only.Don't worry we'll show a way to make an environment variable in windows as well.
You can set an environment variable by running something like:
export PRIVATE_KEY = 0x63b38f9ed50c65b9e48595d8a4b0e398559d2317d3d20ff05162895e4b6ef547
Now if you type echo $PRIVATE_KEY, it'll display the private key.
Setting environment variables with windows the process that we're going to do is actually a little bit different.You can check this link for setting a environment variable in windows.
Limitations of Exporting Environment Variables
It's important to note that this export method we're doing here for creating a environment variables only works for the duration that our shell is live.If we were to close out of our shell and then reopen it, our environment variable that we set would be gone.We'd have to re-run that export command.
We're gonna show you a way to set environment variables so that you don't have to keep doing that.
It's also not great to have in a plain text on your computer.However it's alot better than hard coding it into our script.
Private key PSA
Remember if you're using an account that has real money in it which I highly recommend you don't do, don't send this environment variable or private key or any of the code anywhere because then people can steal all of your funds.Once we move to brownie we'll show you more effective way for private key management but for now be cautious here.If you've followed along and set up a brand new account that has no real money and only test that money in it then great who cares because it's test and fake money anyways.
Accessing Environment Variables
We can actually access the environment variable in Python.To create an evn variable:
export PRIVATE_KEY="your_private_key_without_double_quote"
to view the key
echo $PRIVATE_KEY
to retrieve using python:
**.env file, .gitignore, pip install python-dotenv **
Other thing we can do is create a .env file.A .env is typically where people store environment variables.It's important to not push this to source if this is what you're going to do.In this .env file in python, we could do:
export PRIVATE_KEY="0xyour_private_key_without_double_quote"
If you're going to do this way always set .gitignore and make sure .env is in there.This will help make it harder for you to accidentally push your .env folder or file to github.
load_dotenv()
Python actually has a way of loading directly from a .env file without having to export on our environment variables or run source.env or export or really anything.We can do it with python.env package.
If we close our shell and reopen it, run echo and the saved variable we'll get none.If you run python deploy.py, it's print none as our private key.However we can use dotenv to pulled it directly from our .emv.We could do:
pip install python-dotenv
load_dotenv() looks for .env file and automatically imports it into our scripts.
If we print our signed_txn, we could see a signed transaction.This is exactly what happening when we were looking back at public private keys.We are signing a transaction that is actually deploying a contract to the blockchain that anybody can easily verify.
Sending the signed Transaction
We finally have our signed transaction.We want to send this to the blockchain so it can actually deploy.
Deployment
This will send our transaction to the blockchain.Now if we look at our local ganache and we look at transaction right now, we can see that a transaction actually did go through.
Block confirmation(wait_for_transaction_reciept)
One other thing that's really good practice whenever sending a transaction is we wait for some block confirmations to happen.
This will have our code stop and wait for this transaction hash to go through.
interact/work with thee contract
We've deployed the contract but how do we actually interact and work with the contract.Let's start doing that.When working with contracts, working on chain, we always need to things.
- Contract Address
- Contract ABI
Address & ABI
We need to make new contract object to work with contract.Let's go ahead and create that SimpleStorage contract so we can actually interact with it.
We've address and the abi we can start interacting with the contract exactly as we did in remix.
Retrieve() , Call & Transact
Let's do a print statement to get that initial value that is returned from our retrieve function.It should be initialized to zero.
We get this function retrieve bound to in these parentheses.What's going on?
Call Vs Transact
When making transactions in the blockchain, there's actually two different ways that we can interact with them.We can interact with the call or with a transact.When we use a call, this is just to simulate making the call and getting a return value.Calls don't make a state change to the blockchain.It's similar to how in remix we would call the blue buttons and nothing on the blockchain would actually change.We can actually also call orange buttons and just not actually make a state change.Remix defaults the blue buttons to be calls and orange buttons to be transacts.In python we can actually pick which one we want to do.
A transact call is when we actually make a state change and this is when we actually have to build a transaction and send a transaction.You can always just call a function no matter what that function is but just keep in mind you won't make a state change.You can also always transact on a function even if it's just a view and this will attempt to make a state change.
Something like retrieve even if we transact on it, it won't make a state change.
If we run this, you'll see we do get the zero because now we're actually calling the transaction.
Store function
We've our initial value for our retrieve function.Let's try to update the age using store function.
We know that the store function is orange and we'll make a transaction but if we wanted to we can even just use call on it.
When we send this,it'll return a blank because the store function has no return typr.If we give it a return(uint256) and return an age, you'll see the age returned back.
If we go to the ganache you'll see that we keep making whole bunch of different contract but none of these are contract interactions.That's because we call a function.We just simulate working with it.If we call retrieve again right afterwards, you'll see that it's still zero.
Creating Transaction(Store_transaction)
Let's build a new transaction to actually store some value into this contract.Since we want to make a transaction, we gotta go through the same process as we deployed the contract.
Signing Transaction(signed_store_txn)
Now we've the transaction let's go ahead and sign it.
Sending Transaction(send_store_tx,tx_receipt)
Then ofcourse we need to send it.
Deployment
Let's run this.Alright we still have the print function printing out the current value of retrieve.Let's go over to ganache and see if there's anything different here.
There is instead of all the contract creations, we now have a contract call.If we call our retrieve function again, it'll print out new updated value.
ganache-cli
Ganache UI is really nice because we can see alot of things that are going on.However it's a little tricky to do alot of progammatic stuff.Oftentimes engineer will use what's called a command line interface of ganache.
We're going to use the ganache-cli and this is what brownie is going to use on the backend when we move to brownie.Let's learn how to do that.
install Nodejs
In order to use the ganache-cli, first thing we need to do is download node.js.You can come to this download page and choose your OS and download it accordingly.
install yarn
Next we're actually going to install yarn.Yarn is a package manager.similar to pip which allow us to actually download pieces and packages like the ganache-cli from the package repository.
We can install it with:
On Debian or Ubuntu Linux, you can install Yarn via our Debian package repository. You will first need to configure the repository:
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
Then you can simply:
sudo apt update && sudo apt install yarn
Then we want to install actual ganache-cli.We're gonna be installing it with yarn.so to install this:
yarn global add ganache-cli on a terminal.
This will install ganache-cli as a global command in our terminal.We can test to see if we've done it right.
Run ganache cli , ganache documentation
We can test the ganache-cli by running ganache-cli --version.
If this didn't work our for you then you can install using npm:
first install npm: sudo apt-get install npm
then install ganache-cli: npm install -g ganache-cli
If you've your ganache UI open, please close it otherwise it'll conflict.So to run a local blockchain from the command line all you need to do is :
ganache-cli
and the node will start running directly in the terminal.If you scroll up, you can see lots of familier pieces.
We can see the available accounts just like on the UI and whole bunch of different private keys.This ganache spins off with bunch of random addresses and random private keys.If we wanted to always spin up with the exact same private keys so we don't have to update our private key every time, we can do :
ganache-cli --deterministic
This way we'll always get the exact same private keys and the exact same addresses.You can check out the documentation to see bunch of other flags that you can use to run this and you can see it's listening on 127.0.0.1:8545.127.0.0.1 is also known as loopback address or local host.
update privatekeys,addresses,http provider
To work with ganache in the command line all we need to do now is update our private keys and our addresses.Let's also update http provider.
open new terminal & deploy
Let's open up a new terminal and run python deploy.py
You can see the exact same output as we got when working with the UI.If we flip to the node terminal, you can see different calls to our blockchain.
Each one of these calls is a specific json rpc call to our blockchain that we're making to interact with it.We can also see the information of the transaction that we send.
deploy to testnet/mainnet
How do we actually deploy this to testnet or mainnet? When we were working with remix, all we had to do was switch to injected web3 and we used our metamask as our blockchain connection.Well in our script we don't have metamask natively with our script.So we need some way to connect to the blockchain.We can see that when we're connecting to our own blockchain we just use a rpc url that connects to our local blockchain.
To connect to the testnet or mainnet, we can exactly do the exact same thing.All we have to do is swap the rpc url with the url that connects us to a mainnet or a testnet.We can also run our own blockchain node similar to how we're running our own local blockchain node.
We can run a node that actually connects to a real blockchain.However it's not always practical or really easy to do this.So sometimes we wanna use an external or third party client that actually run a blockchain for us.
Infura, Alchemy
Let's learn a little about Influra.It's an application that'll give you a blockchain url for you to connect with for you to run whatever you want to run and you can get started for free.Let's go ahead and register.
There's other services out there that you can also check out like alchemy which is another fantastic blockchain as a service platform.
Create project
Click create new project and name it brownie.We'll have a whole bunch of project keys and project secrets.We will also have endpoint section as well.This is how we're going to be deploying to the different networks.
update the rinkeby url, Chain id , address & private key
We copy the url and back in our script all we have to do is swap rpc provider with a new url we just copied.
We also have to change the chain id, our address and the private key.If you ever are confused as to what is the chain id of the chain that you're working on, you can always check here.
The address and the private that we gave it aren't gonna have any testnet.We need to go in our metamask and grab the address and place it in for address.Then account details, export private key, grab the private key and past it in our .env file.
Since I've my private key stored as an environment variable, I need to run source .env so that my private key is now updated.
The reason we're using metamask address and private key is since we're making transactions to a testnet, we need some testnet eth.
Now we've everything updated for deploying to rinkeby.Let's go ahead and run the script now.It'll take a lot longer but we can see we got the exact same responses.
If you take the address and go to rinkeby etherscan, you can verify what just happened.
Summary
We've learned alot about python, deploying our own local blockchain, deploying to a testnet and mainnet, working more with private keys, creating transactions, signing transactions and then sending transactions.
Lesson 5 - Brownie Simple Storage
There's going to be alot to actually managing all the contracts that we work with having to write our own compile code, storage code is going to take alot of work and what if we wanted to interact with one of the contracts that we deployed in the past.Well we'd have to keep track of all the addresses and manually update our address features with an address.Maybe we didn't want to deploy a new contract every single time, maybe that we want to work with contract that we've already deployed.What if we want to work with a whole bunch of different chains like Rinkeby, mainnet, our own local net?There seems to be alot to manage here and we still haven't talked about writing tests.
**Brownie Intro & Features **
This is where brownie is going to come into play.Brownie is currently the most popular smart contract development platform buit based on python.It's used by defi giants like yearn.finance, curve.fi and badger.finance and each out having billions of dollars currently locked in value.
The reason that we learned a little bit about web3.py first because brownie heavily relies on web3.py.So let's all again but in brownie and we'll see how much easier it is to actually interact with.
install Brownie
Brownie is incredibly powerful and makes our lives fantastically easier.So get ready to learn one of the most powerful tools in the smart contract developing ecosystem.Let's go ahead and open up our terminal and let's get started installing brownie.It's recommended to install brownie via pipx.Pipx installs brownie into a virtual environment and makes it available directly from the command line.Once installed you never have to activate a virtual environment prior to using brownie.To install with pipx we can go ahead and run:
python3 -m pip install --user pipx
Once we run that we can run:
python3 -m pipx ensurepath
Then we'll want to close the terminal by hitting little trashcan and then re-open it.And then:
pipx install eth-brownie
and one more time we're going to close and re-open the terminal.
You can tell you've brownie installed correctly if you run
brownie --version
1st brownie simplestorage project
Let's create our first brownie project.We're going to be using exact same SimpleStorage code that we just went through except for we're going to use it in brownie.To create a sample folder with everything we need with brownie, we can just run :
brownie init
We'll get a new brownie project initialized in the director that we're currently in.If you type ls, you'll be able to see all the folders that are created or you can just seem them on your side panel in vscode.
Brownie Folders
Let's talk really quickly about what each of these folders is going to do.The build folder tracks alot of really important low-level information.It's going to track any interfaces that we're working with or deploying.It's going to keep track of all of our deployments across all of the different chains so we no longer have to manage that ourselves and it's going to store all the compiled code.
Rememer how in our SimpleStorage code we actually saved everything to compiled_code.json.Well brownie is actually going to do all of that for us into this build/contracts directory.So we can always reference it later.
The contracts directory outside the build folder is where we're going to put all of our contracts.Brownie knows to look inside of this folder when looking for new contracts to compile, deploy or anything else.
Interfaces is where we can save and store different interfaces.Remember how when we're working with chainlink, working with interfaces makes it really easy to interact with a blockchain application.
Reports are to save any type of report you run.Scripts where we can automate tasks like deploying, calling different functions or really anything we want.
Then we've a 'test' folder which is incredibly powerful and we're going to be using alot.We also have .gitattributes and .gitignore which are helpful when working with version control like git.
So let's go ahead and start working with brownie and really understand what's going on here.Let's add our SimpleStorage contracts to the contracts folder and copy and paste the code from the SimpleStorage that we've been using whole time.
brownie compile & store
Now that we've a contract we can already start working with brownie and even compile code without even having to write or work with our own compiler.All we need to do is :
brownie compile
Brownie will automatically read the version of the solidity and store all of the compile information in this build folder.If we go to contracts, we've SimpleStorage.json and there's alot of familier pieces here like abi, opcodes section and alot of useful information.
Great we've already compiled our smart contract.So why don't we actually deploy this to blockchain?
brownie deploy
To do this we've to write a script which will allow us to do whatever we want.We're going to create a new file and we're just going to call it deploy.py similar to last time inside scripts folder.
Brownie can run scripts by running brownie run.
If you want to take a quick minute to familiarize yourself with all the different commands that brownie has just run brownie
We can define that we want to run deploy.All we have to do:
brownie runscripts/deploy. py & default brownie network
And we can run:
And as you can see it automatically does this launching thing.Brownie defaults to always working with a local ganache-cli blockchain.It's running the exact same command that we ran earlier and it has a bunch of different flags like accounts 10, certain hardfork, certain gasLimit etc.So at the beginning of all our scripts if we don't give brownie a network to use, it'll spin up a local ganache and at the end of the script it'll tear it back down.
Typically what I like to do is put all of the logic of our deployment in its own function.
brownie Advantages over web3. py in deploying
In order to deploy our contract let's look back at our web3.py version of deploying this.First we compiled it but brownie does that automatically.Then we dumped into a file but brownie does that automatically.We got a bytecode and an abi but brownie does that automatically.We added a local blockchain to use but brownie autmatically spins up a local ganache.We do need an address and a private key.
getting address & private key using Accounts package
How do we actually get our private key and our account into brownie.Brownie has an account package that actually natively understands how to work with accounts and we can import it into our scripts.
with this account keyword we can add an account a number of different ways.If we're going to work with local chain as you saw earlier 'ganache-cli' will spin up 10 fake accounts for us and brownie automatically knows that we can work with that account.
add default ganache account using index
We can do something like:
We're going to take the account that spun up at the 0th index because the accounts object is just an array.So if we run this now: It's going to spin up us an address and a private key that we can just work with without having to define a private or do anything.It does all of that for us.
We do still want to know how to add our own private keys so that we can work with the testnet.When we're working with a development network or working with brownie's automatic ganache-cli.If we want to work with testnet though, we've to do something else.
**add accounts using Commandline **
Another way to add your accounts in brownie is to use the command line and actually add them natively into brownie.We can do :
brownie accounts new account_name
This will then prompt us and say enter the private key that you wish to add.Let's go ahead and grab our private key from metamask and we add 0xand the private key.
Brownie will actually password encrypt your private.After we give a password, we've a new account natively integrated into brownie.To see it we can do: brownie accounts list and we'll see we've our SS_account with address.
We can get rid of testing by: brownie accounts delete SS_account
If we want to work with SS_account that we've added to brownie via the command line, we can get it with:
This time when we run the script, it's going to ask us the password.We need to enter the password to decrypt the account.
Note: If we're going to talk about safety and you want to safely secure your keys, this is one of the safe ways to do it because it's not going to store it in git, you're not accidentally push it up to github or show it to anybody and it's going to be password encrypted.
Oftentimes you're going to want to add to do a mix of working with the local ganache ones and your own keys.We'll learn how to flip back and forth between them in a little bit.
add accounts using env variables
The third way I like to use is still again using an environment variable script.Oftentimes it's really easy just to have your private key be an environment variable.This way you won't have to keep putting the password in every single time you run a script.It's a little bit less secure and just another tip for myself:
Never put private keys associated with wallets that have real money in them as environment variables or in a .env file.
Let's go ahead and create an environment variable file.Brownie has an additional feature that allows us to easily work with environment variables in an environment variable folder.We can tell brownie to always pull from our .env file in a brownie-config.yaml.It's a special file that brownie always looks for to grab information about where you're going to build, deploy and grab things.
In the yaml file all we need to do is:
dotenv: .env
This is telling brownie to grab the environment variables from the .env file and what we can do is.
adding wallets in yaml file and updating in account
This seems to work perfectly but I like to make this method even more explicit.We're going to take this version that we just learned and improve it.
In our brownie-config we can actually add more information about what wallets we want to use and when we wanna use them.
In your yaml file if you surround a string with $ sign and some curly brackets, it'll automatically get transformed into the environment variable.So if we go back to deploy, we can actually change the code:
The reason that this is better because now we've one canonical place where we're always going to pull our private key from.Instead of having to go through all of our scripts and update it based on whatever we can change an environment variable.
For now let's just stick using accounts[0].Since we wanna just use the account that brownie makes for us with ganache.
importing contract simplestorage
Brownie is really intelligent and we can actually go ahead and import contract directly into our script.Web3.py version we opened a contract and read from it and that's how we're able to interact with it after we deployed it.
In brownie what we can do is:
SimpleStorage is just the name of the contract.
importing & deploying in brownie vs web3. py
This is how we deploy to the chain.Anytime you deploy to chain or you make a transaction, you always need to do a from and say who you're going to deploying from; what's the account that's going to deploying this.
As you can see this code of just deploying is much quicker than what we did in web3.py.In web3.py we'd to get the bytecode and the abi, nonce, create the contract, create the transaction, signed the transaction and then send the transaction.
Remeber how I said before you could either make a transaction or a call.Brownie is smart enough to know whether or not what you're doing is going to be a transaction or a call.In our case since we're deploying a smart contract brownie's smart enough to know that we want to make a state change so let's make a state change.
What happened was brownie again per usual launched a local ganache chain and then sent a transaction to deploy SimpleStorage.It says SimpleStorage deployed at and the address it was deployed at.We can see how much quicker this is to actually deploy.
recreating web3 .py script in brownie
Let's go ahead and do exactly what we did with web3.py.Let's call initial retrieve function and we'll update the age with a new value of 15.
Since this is a view function, we don't have to add from account in here.We know that retrieve is a view function so we don't actually have to make a transaction.
Now let's try updating it:
Since we're doing a transaction in brownie, we always have to add who we're going to transact from.In our case it's from: account.Similar to web3.py transaction.wait for how many blocks we wanna wait.
tests
Running these scripts is fantastic but we need a way to actually automate that our contracts are doing what we want them to do.We don't want to always manually check that all of our stuff is doing what we want to do.We don't want to have to manually check that 15 is actually updating appropriately.This is why running tests are so important and automating your tests is going to be crucial to becoming a successful smart contract developer.You can write tests directly in solidity and is a great way to actually test your smart contracts is to learn how to do it right in solidity.
However alot of professional developers code their tests in the smart contract development framework language like python or javascript.Doing it in this way allows you to get more flexibility and customization with what you're doing with your smart contracts and not being confined to whatever only solidity has.
So let's go ahead and learn how to actually write our smart contract tests in python.This is why the test folder is there.Create a new file called test_SS.py inside tests folder.Make sure you do add test to the front of the filename because this is the syntax that pytest is going to be looking for.
Then in our test we can set it up the exact same way we set up our deploy function.
from brownie import SimpleStorage, accounts
We can start defining our tests.We want to test to see that when we deploy our smart contract that it gets started off with zero in that retrieve function.So we'll create our first test.
Typically testing in smart contracts or testing really in anything is going to be seperated into three categories.
- Arrange
- Act
- Assert
We're going to bounce around and be a little bit loose with the definition.However keep in mind this is the setup you wanna use.Later smart contract examples we're going to go through a much better testing setup.
In our arrange state we're going to set up all the pieces that we need to get setup.
- In our arrange state, we're just getting our account so that we can actually make contracts.
- In out act stage w're going to deploy SimpleStorage contract, call the retrieve function to see what it's starting value is.
- We're going to compare to see if that starting value is what we expect.
We can test this with:
brownie test
Let's go ahead and test updating it with 15 and see if it works as we want it to.
If you wanna test just one function use -k function_name: brownie test -k test_deploy
You can run brownie test --pdb and add something wrong in the script.Once it kicks out and is wrong, will actually put into python shell.Now we can check to see the variables of the script.
Another important flag is -s.It tells exactly what's going on and if we had any print lines, it'd print lines as well.
Everything you can do with brownie test is actually comes directly comes from pytest.So if there's some flag you wanna use or some awesome debugger you wanna use, you can use it with brownie just by looking at the pytest documentation.
Deploying to a Testnet
We've a script to deploy, have some tests and our contract.This is great but we want to deploy to a testnet.How are we going to do this?
Let's look at back how we did with web3.py.We just needed to add http web3 provider, add our address and our private key.Brownie comes pre-packaged with a list of networks that it's already compatible with.You can see all of the networks by:
brownie networks list
Something important to note is that there's a difference between the development networks and the ethereum networks.Whenever we deploy to a network, we default to the development network.Any network that you see under this development section is going to be a network that is temporary.These are networks such as the temporary ganache network that brownie automatically spins up when we've run any of the scripts we've written so far.
However the ethereum ones are going to be our persistent networks.Brownie is going to keep track of our deployments and keep track of everything in there.
In our web3.py we used an rpc url or an http provider from an infura to connect to a testnet.We're going to use that exact same methodology here.How do we actually get this rpc url into our brownie smart contract package?
Well one of the easiest ways is with an environment variable.Brownie actually already knows that infura is a thing and can look natively right away for infura.
We used our project id from Infura.If we do brownie networks list, we can see any network that infura has access to.You can see infura in brackets.These are networks brownie will actomatically know about if we're working with infura.
If we wanted to deploy to Rinkeby, we could just run:
brownie run scripts/deploy.py --network rinkeby
We got an issue here because accounts[0] only works when brownie works with ganache-cli.We've to use our actual private key here and that's where some of the other versions of working with private keys is gonna come into play.
Sometimes I'll even add a get account function.
Once we've deployed to a blockchain, you'll see our build contract will actually change.Our deployment folder will have a new deployment.Everytime you deploy to a blockchain brownie will save that deployment so you can always go back see what happened with that deployment.You'll notice it's seperated by chain id.Anything that's in the development section isn't going to be saved to the deployments area.We can actually interact with contracts that we've already deployed onto a chain.
So go ahead and add new file inside scripts called read_value.py.This code will read directly from the rinkeby blockchain and it's going to read from a contract that we've already deployed.Remember we did something similar in web3.py by using the address and the abi.We're going to do the exact same thing but in brownie.
How do we actually interact with the SimpleStorage contract that we've already deployed?Well SimpleStorage object is actually just an array.If we were to print SimpleStorage, we'd get:
We can access the different indices inside of it.What if we do print SimpleStorage[0]? we'll get the address and if we check on etherscan, we can see that address is indeed the contract that we just deployed.Brownie knows that we just deployed it because again in our build section in the deployments on the rinkeby chain which has the chain id of 4, we have the contract that we've deployed.
Now we can directly interact with the contract.
Remember how I said whenever we work with a smart contract we need to know it's abi and it's address.Brownie already knows the address and the abi which it got saved into deployments folder and json file respectively.
Brownie console
Now that we know how to write some scripts, deploy things with brownie and work with brownie, I'm going to show you one of the most powerful features to also work with brownie.
Typically we write our scripts when we want something to be reproducible and we want to do something over and over again.Deploying SimpleStorage or reading a value is something that we're probably going to want to do over and over again.However maybe we want to work with some of these contracts a little bit ad hoc and get into a shell where we can actually interact with these contracts.This is where brownie console is actually going to come into play.
We can run brownie console
Brownie will actually kick us off into a console.It has all of our contracts and everything already imported.Everything that is improted via brownie in our scripts is automatically already imported into the shell.We can go ahead and even deploy our SimpleStorage contract.We can take the deploy line and paste it on the shell and can see the deployed contract.We can do everything that we normally do in Python.
Lesson 6: Brownie Fund Me
All right let's move on to our next project.We're going to take the fundme contract that we made recently in remix and actually import that one into brownie.With this we're going to go over a lot more advanced features that brownie has to offer us and we're getting into some more sophisticated testing and deployment mechanisms for this.
Initial Setup
I'm going to make a new directory mkdir brownie_fundme
After you're inside brownie_fundme project, initialize a new project.
brownie init
We're going to create a new contract called FundMe.sol which is going to be exact what we had before for our FundMe contract.Just copy past the code.
Typically this is where in our last project we just ran brownie compile.
What happens if we run brownie compile here? Brownie is going to give us error saying the source wasn't found.Well remix understands that @chainlink/contracts is an npm package that it can import from however brownie isn't aware of npm packages and brownie can't actually download directly from npm .However brownie can download directly from Github.So we've to tell brownie where it should be downloading external third party packages from so that we can use them in our contracts.
Let's change the global compiler version to 0.6.6 and the compiler version to ^0.6.6 so that our linter is happy.
We need to tell brownie where to import those @chainlink/contracts... from.From github instead of npm.Intentionally there's a package out there specifically created for downloading chainlink contracts.
Dependencies
The way that we can tell brownie where to get these contracts is in our brownie config.In here we're going to create a dependencies section and this is where we tell brownie:
So to get these chainlink contracts, we can look up here.Instead of downloading from npm, we're just going to download directly from the github repo.We need to tell brownie the organization(smartcontractkit), repo_name(chainlink-brownie-contracts) and the versions.
It does look like the latest version went backwards but as of a recent release as a 1.2.0 version of the package, all the releases of the chainlink-brownie-contracts are going to match exactly the @chainlink/contracts npm tag.So that's why it looks like an earlier version.
So now brownie knows to download this repo from Github.Once we do that everything in the contract section will be available for our solidity code to import and use.However that's not the final story here.We also need to tell brownie what @chainlink thing means because we're downloading from smartcontractkit/chainlink-brownie-contracts but what's that chainlink thing.
Remappings
So we need to tell brownie whenever we're using @chainlink we're actually referring to smartcontractkit/chainlink-brownie-contracts@1.1.1 this import.We need to tell the compiler this.Let's add a new section.
Let's go ahead and try to compile.We'll see it compile successfully and if we look at our build folder in the contract section we've new folder called dependencies.It downloaded AggregatorV3Interface and SafeMathChainlink because we're using both of those files in our solidity code.
Let's write our first script to deploy to a development ganache chain(brownie's built-in chain).
Rinkeby Deploy Script
Let's go ahead and create a new file(deploy.py) inside scripts.We'll build a simple script to work with rinkeby and then we'll actually learn how to deploy this to our own local ganache development chain.
Let's build this deploy_fund_me function.So first we're going to need to get an account and same as last time we used a function called get account which would know to switch back and forth between if we're on development or if we're with an actual testnet that we could pull from our config.We can copy that function and use that in our script here.
What I like to do with this get_account() is add it into it's own file called helpful_scripts inside scripts folder.In that file I'll add that get_account function.
And in our config we're going to have to add wallets and from_key.
Now that we've added get_account() to it's own scripts.How do we actually use that in our deploy script.
init.py
Depending on the version of python that you're in you might actually have to create a new file first named __init__.py.You might not bust just incase let's make it inside scripts folder.With this __init__.py python knows that it can import from other scripts and other packages in this project.Now we could do:
We're importing get_account function from our helpful_script.We gonna make get_account function a little more robust a little bit later but for now it'll work perfectly for what we're looking to do.
Deploying to Rinkeby
Now we could just run our typical deploy function.
brownie run scripts/deploy.py --network rinkeby
Contract Verification
Our contract is deployed but if we look at our contract, it's kind of like blank jarble of bytes.Nobody can actually see the contract and easily interact with it.We want to make our contracts easy to interact with.
The mannual way
What we can do on etherscan is verify and publish our smart contracts.To verify our smart contracts all we've to do is hit that verify button, choose solidity compiler type which we know is going to be 0.6.6 and we know we're working with MIT license.Now to continue we then have to add in all the rest pieces.We'd have to do optimization, enter our solidity contract code which importing fundme like we did wouldn't actually work because etherscan dowsn't know what @chainlink/contracts is.We'd have to copy paste the code from these imports to the top of our contract.Removing the imports and copy pasting the code associated with that files is known as flattening and this is an important concept for verifying our smart contracts on platforms like etherscan.
Programatic way to verify
However brownie has a really nice way to get around this actually.What you can do is go to their main site, you can sign in and sign up and get an api key.You can go to My Profile and scroll down to API-KEYs.This is where we can create an API key for us to interact with and programmatically verify our smart contracts on etherscan.
Create an API Key
Hit add, give name which will create an api token that allows us to verify our smart contracts.What we'll do to use that token in brownie is we'll copy that api key and we'll move back to our scripts.Sets that API key as an environment variable.So we'll do:
export ETHERSCAN_TOKEN=*your_api_key*
Now to verify this all we've to do then is once we deploy this contract we'll tell brownie whether or not we want to verify the contract.
If we re-run this script let's see what happens?
We'll see waiting for an api-rinkeby.etherscan.io to process contract and we're submitting our code for verification.It'll say verification complete: pass verified, where the contract has been deployed and what's been done with it.
Now if we go back to rinkeby etherscan, paste the address, we now have a little checkmark associated with our contract.If we click contract we can see all of the code for our contract.If we scroll up the code we can see they indeed flatten the contract.We can even read the contract, see the different public variables such as the getVersion, getPrice and all the exact same buttons that we saw in remix.The read contract section is for the view function the ones that aren't going to be making a state change.Write contract is when we want to call something like fund or withdraw and we could even interact with it.
For example we could connect to web3 metamask and it'd automatically get connected to our metamask and we can call fund and withdraw just like we did in remix.
Sometimes verification process can be a little bit touchy and for version of solidity 0.8.4 there's currently a bug that makes it verifying a little bit tricky but it's being worked on.
Deploying to ganache
We've written a deploy script for deploying directly to rinkeby however as you know we're always going to want to be able to deploy to our own local blockchains or brownie's built-in development chain so we can test alot quicker and we can write some tests.
We've a little bit of an issue here.The first issue being here is that our fundme contract currently has an address hard-coded to work with the rinkeby chain.So infact the way it's written right now it's going to be hard to work with any other chain other than rinkeby.The second is that the price feed contracts don't exist on a local ganache chain or a ganache chain that brownie spins up.There's two ways we can get around this:
- Forking
- Mocking
We can do what's called forking and work on a forked simulated chain or we can deploy a mock or deploy a fake pricefeed contract on our ganache local development chain.
Mock
Deploying mocks is a common design pattern used across all software engineering industries and what it applies doing is deploying a fake version of something and interacting with it as if it's real.Right now if I run brownie run scripts/deploy.py and I don't set the network flag, we're going to actually have a default spinning up a ganache chain.It's even going to try to verify it which it's going to run into an issue because we can't verify on a ganache chain.
So we've a couple issues that we need to address here in order for us to get this work on a ganache chain.
Similar to how in get_account function in our helpful_scripts, we do a little bit of checking.If we're on a development network, use the developments version otherwise pull from our config.We can do exact same mentality but working with mocks.
So the first thing we need to do is we need to parameterize our fundme solidity smart contract so that we don't have hard-coded address.
Constructor Parameters
Right when we deploy the contract we'll tell it what price feed address it should use.Right when we call our deploy function instead of having hard-coded and add this parameter to our constructor.
Instead of creating AggregatorV3Interface contracts right in the functions, we're just creating a global one.
And right in our constructor when we create it we'll set:
Now what we can do is just delete AggregatorV3Interface part from getVersion and getPrice function.We can do that exact same thing in our constructor when we deploy the contract.
We can even verify running brownie compile.
Our deploy function is going to need to look a little bit different.We're going to need to pass price feed address to our fund me contract.To do this all we've to do is paste that address before we've our {"from":account}.
This is how we can actually pass variables to constructors.Anything inside constructor function you can pass through brownie in our deploy script like we did.We could always just pass the rinkeby address but that's not really going to solve our problem obviously right.The problem is that the we always have the rinkeby address when we say if we are on a persistent network like rinkeby use that address other wise deploy mocks.
We'll talk about mocks and get into them in a second.But first let's set it up.
We can check what network we're on by importing from brownie network bit.
In our brownie-config we can add different addresses for different networks.
Now what we can do in our deploy script is if we're not on a development network let's grab the address from our networks section and we also need to import config in our deploy script.
What if we're on a development chain?Well we're going to have to deploy a mock.Right now on all these live networks that we're working with there's a version of the price_feed_address.Our development chain obviously there won't be one because it's going to start off as blank.What we can do is we can deploy our own version of the price_feed contract.This is known as mocking and we can interact with it accordingly.In order for us to deploy price_feed contract ourselves we're obviously going to need to have the solidity code associated with it.
In our contract section we'll create a new folder called test.When you create a test folder in your contract folder, this is typically where mock contracts are going to go and we'll create a MockV3Aggregator.sol file inside test.This is where we'll add the code for us to deploy our own price feed.I'm going to show you another repo pretty soon called the chainlink-mix that we're going to work with.For now if you're looking for one of these mocks I highly recommend just pulling it from the chainlink-mix repo.We can grab it by going to contracts/test and grabbing MockV3Aggregator.sol.We can just copy the whole code and paste it into our MockV3Aggregator. This has all the same exact functions as a real pricefeed contract such as decimals and latest answer which we're going to be using the most.We can also see it's constructor which it takes variables whenever the contract is deployed.Once we've this contract in our test section we can run brownie compile and brownie will go ahead and compile it.
Same way we import FundMe we'll be importing MockV3Aggregator.We setted decimals to 18 and initial answer to 2000.Since we're deploying we need to send the address.Let's go ahead and even try this on a development network.We could do:
brownie run scripts/deploy.pyWe don't need a network flag here.
We'll spin up the ganache and we even will deploy a mock aggregator contract first then we'll do our fundme part.Of course we're running to an issue because we're trying to verify a contract on a chain that doesn't exist.Etherscan doesn't know about our local ganache chain.To fix this instead of doing publish_source=True, we can have this published source be again based on what chain that we're on.We'll go to brownie-config.
Back in our deploy scripts we now say publish_source is going to be pulled from our config:
Now if we run this again we'll spin up our ganache, deploy a mock pricefeed contract, deploy our fund me contract and it completes successfully.We have a way our fund me contract that uses a price feed contract to our own local development environment with a mock aka a fake pricefeed contract.
Refactoring
This is great but our deploy script is starting to look a little bit big and a little bit clunky.So actually cleans this up to make it look a lot nicer.
First we wanna do is fix big 2000000000000000000000 number.Looking at right away I'm not exactly sure how many zeroes it has.So to make it little bit nicer and more readable we can once again call on our friend web3.py
This toWei function will just add 18 decimals to this 2000.This is much more readable that the inital value for this is going to be 2000.The next bit is that we're always deploying this MockV3Aggregator and if we already have a mock deployed to whatever network that we're working on, we don't need two mocks.
If the length of MockV3Aggregator is less that equals to 0, only then we deploy it.MockV3Aggregator is just going to be a list of all the different V3Aggregators that we've deployed.
Insted of using mock_aggregator.address we can just use
This is going to used recently deployed MockV3Aggregator.Then ofcourse we no longer need to set it as a variable and that looks a little bit cleaner and we'll make this a little bit more efficient.
Deploying mock bit though is going to be something that we're actually going to do relatively frequently.So what I like to do is put this whole mock deploying bit in its own function in our helpful scripts.
We need to import MockV3Aggregator, Web3 and instead of account variable we'll use get_account function.
In our deploy.py we'll place deploy_mocks() in that code and import it from helpful_Scripts.py and we can delete web3 import in deploy.py
This is starting to look lot more sophisticated and this is great because now we have a way to deploy base off if we're on a live chain or a development chain.And one more thing I'd like to do so that everything is parameterized, I'll set MockV3Aggregator parameters as static variable.
Deploying to a persistant ganache
So let's say now that I wanted to deploy to my own ganache instance.Well what we can do is we can open up our ganache here, click quickstart and we'll just quickly create our own ganache blockchain.
Instead of brownie spinning of it's own ganache, brownie is smart enough to detect if you're running your own ganache instance and will automatically attach itself to it knowing that is going to be a development environment.If we go to our ganache we'll see we've two transactions which are going to be contract creations.This is great for testing quickly and locally but there's an issue with this.
For development networks again brownie doesn't keep track of those so in our build folders if we go to deployments we only are saving stuff from the rinkeby chain here.So we need to tell brownie that there's another network that we want you to work with and we want it to remember the deployments to that chain.
Adding network to brownie
We can add a new network to our brownie networks list.We want to deploy to ganache chain and we want brownie to remember those deployments.Well we can add a network in ganache network list.
brownie networks add Ethereum ganache-local host=http://127.0.0.1:7545 chainid=5777
We can see a new blockchain called ganache-local and this is going to reach out to our ganache UI or our ganache CL depending on what we're running.
If I try to deploy deploy.py in our ganache-local, we'll get this issue
Ganache-local isn't development.It's going to go ahead and try to pull from our config file.We don't want this.We want to actually deploy mock for our local ganache if a mock hasn't been deployed.So we can extend our definition of what a development environment is.
In our helpful_Scripts we can add a flag.
We can import this environment into our deploy.py
This is saying whatever network that we're on if it isn't development or ganache-local then go ahead and use config.If it's one of these two, we're going to deploy a mock.
Now we've this let's try this again.
We run into a different issue saying we don't actually have enough gas.
Why's this? Let's look at our get_account function which is directly looking for development chain only.So we could do
if network.show_active() in LOCAL_BLOCKCHAIN_ENVIRONMENTS:
If the network that we're working on is development or ganache-local, return accounts[0].
Let's try this.
Now we're running to this key error of ganache-local.In our config we'll add this new network.
Let's try this one more time.
Perfect mocks has been deployed and our fundme has been deployed.If we look in our build folder in deployments we now have new chain id for 5777 for saving these deployments and it looks like in our ganache chain we've it actually saved in.
Important this to note if you were to close or delete ganache chain, all of your contracts will be lost.You won't be able to interact with them again.
Withdraw and Fund
Now that we've deployed this let's write a script to interact with it.Create a new file inside scripts directory naming it fund_and_withdraw.py.
Since it's going to resemble ethUSD pricefeed, it actually only has 8 decimals because getPrice function has only 8 decimal places.
And for MockV3Aggregator we're going to use those exact values.We're not going to do web3 converting.
I've added a getEntranceFee function to our contract(FundMe.sol).
First we're going to run brownie run scripts/deploy.py --network ganache-local because we changed FundMe.sol
and then we can run brownie run scripts/fund_and_withdraw.py --network ganache-local
and run the fund_and_withdraw.py which will work fine.
Withdraw
You need to return fund_me contract in FundMe.sol.
Let's run the script.You can see both funding and withdraw transaction going through.
This is the script we can run on a main network if we like.
Testing
We're going to move into actually writing those tests and for these tests we're going to want to quit our ganache UI.Let's go ahead and write some of these tests in test_fund_me.py file.
We're on this test we're going to want it to be able to work independent of the network that we're working on.We write a test to see if we can fund or withdrawl.
Default Network
You may be wondering why sometimes we use --network flag sometimes we don't.In our network section of "brownie-config.yaml" file, brownie automatically picks a default network to use and the default is always set to development.However we could set this to be anything we wanted.We could set the default to be ganache-local or rinkeby.Whatever you set for your default in your brownie config is what the network will be defaulted to.
Right now when we run brownie test this is equivalent to writing brownie test --network development.
We can also run the test in Rinkeby but oftentimes we don't wanna tets all of our functionality on rinkeby or live networks because it's going to take a long time for them to run.Sometimes we only want to run tests on our local chains.
We can use pytest skip functionality to do so.To work with pytest we need to install it.So go ahead and run pip install pytest.
To demonstrate this lets create a test that makes sure only the owner can withdraw and nobody else can.
Well we want this to happen.How do we test that we wnt this to happen?We just need to tell our test that we want this to happen.
First we need to import exceptions from brownie.This way we can tell our test exactly what exception we're expecting to see.
If it reverts with the VirtualMachineError that's good.We want it to revert when we try to withdraw from different account.
We can see it passes the test which is exactly we're expecting.
Mainnet fork
Mainnet forking is incredibly powerful when we're working with smart contracts on mainnet that we wanna test locally.Let's talk about forking for a minute.
In a blockchain there's a whole bunch of blocks and huge chain that we can actually work with.All the information is public information.Block gonna have lots of transaction.In addition to transaction, it's going to have pricefeed contracts, aave contracts etc.It's going to have all these different contracts in it.So if hypothetically if its already there, we should be able to basically copy the whole blockchain and do some simultions ourselves and that exactly what forking does.
A forked blockchain literally takes a copy of an existing blockchain and bring it into our local computer for us to work with.We actually have control over the forked blockchain since it's going to run on our local computer similar to ganache.All the interaction we do on the local blockchain are not going to affect the real blockchain becuse it's our local chain.
Mainnet fork is a built in part of brownie and also pulls from infura the same way it works with rinkeby and coven and everything else.We can start to interact with the mainnet fork contracts the exact same way therefore.
We can take the whole rinkeby section(brownie-config.yaml), copy it, paste it and we'll just change eth_usd_price_feed address to it's mainnet address.We'll go docs.chain.link, copy that address and put it on config file.
Since it's going to be a fork, verify needs to be False.
If you try to run one of our scripts like brownie run scripts/deploy.py --network mainnet-fork, you'll see we'll actually run into an issue saying insufficient funds for transfer.
In our deploy script when we do our get_account function call and in our helpful scripts right now we're using accounts.add[config["wallets"]["from_key"]) which our account has 0 money in it on mainnet.So we need to tell brownie when we're working with mainnet fork it should create us a fake account with a hundred ether.However we don't want it to deploy a mock because the price feed contracts already exist but we do want it to get an account.
Now if we run this we'll still get an error saying index out of range.Brownie built in forking mechanism doesn't actually come with it's own accounts here.
Custom mainnet-fork
I like to create my own custom mainnet fork right in brownie.
brownie networks add development mainnet-fork-dev cmd=ganache-cli host=http://127.0.0.1 fork=https://eth-mainnet.alchemyapi.io/v2/ISvH6u9dX3dloFAfYbic-eIht63naI20 accounts=10 mnemonic=brownie port=8545
command to run the fork is ganache-cli fork is going to be from alchemy instead of infura because performance wise forking from infura pretty much always give an issue.So go ahead and create account in alchemy and create a project, view details and view key to copy http address to fork it.You'll know you've done it right if you can something like:
In our config we'll change mainnet-fork to mainnet-fork-dev and in our helpful scripts we'll add mainnet-fork-dev without removing mainnet-fork.
Now our get_account function should return accounts[0] for mainnet-fork-dev and it should actually work.
And we should be able to test exactly the same way.
Adding to Github
We're going to teach you how to actually share you code and enter the world of open source and decentralized code.First initialize you repository by:
git init -b main
and add your username and email to our config
git config user.name "demo"
git config user.email "demo@gmail.com"
We're going to do couple of things to push all of our code to github however we don't want to push .env.Add .env in .gitignore file.This will help us prevent accidentally pushing our .env file to Github.
We can choose what file we wanna push by doing
git add .
and if we do git status, it'll show us all the different files that we have staged to push to github.
To remove .env if accidentally pushed
git rm --cached .env
Add .env back to .gitignore file and do git status and git add .
Now we commit by
git commit -m "First commit"
Now we can add the folder to the Github by:
git remote add origin *repo url.git*
git push -u origin main
You'll see all of your files in Github repo.
Where should I run my tests?
The default for every single one of your contracts is as follows:
- Always 100% need to have tests that pass on a local brownie spun up ganache instance.This means you need to deploy mocks.
- Testnet: Always (but only for integration testing)
- Brownie mainnet-fork: Optional
- Custom mainnet-fork: Optional
Testing on mainnet fork can be done and should be done whenever all of your contracts and all of your interactions are going to be on chain, on one chain without any external off-chain components.
- Self/Local Ganache: Not necessary, but good for tinkering
Now let's move to our most challenging example.Once you complete this example you basically will have all the tools to be an incredibly powerful smart contract developer.Then after that we're going to show you the chainlink mix package and how to do what's called a brownie bake to automatically open up this package with all these scripts and all these contracts pre-built in which makes easier and faster for deploying our smart contracts.However let's go through the process of understanding all the pieces that are going to be inside of this brownie mix.
Let's get into our most advanced smart contract project that we've made so far.This is going to be the best example of a full scale application. By full scale I mean end-to-end full suite of our brownie.
Decentralized Lottery
What we're going to do is create a lottery application where anybody can enter the lottery and a random winner is selected.So let's do it and let's get started.
Setup
First of course we're going to make a new directory.
mkdir smartcontract-lottery
and open that folder in visual studio.
Let's go ahead and start a new project.
brownie init
and create a quick readme.md file to explain what we're trying to do here.
Wait..is this really decentralized?
Since we've an admin here this means that our application isn't necessarily going to be truly decentralized because we've a single person chooses when the lottery is over.
Could we make it decentralized?
We could scale this out to have maybe a dow being the admin or something like that.
Chainlink Keepers
Or we could have the lottery automatically open and closed based off some time parameters.
But for the moment this is the setup that we're going to have.
Lottery.sol
First thing that we're going to get started with of course is our lottery contract and let's begin with our initial setup.
Main functions of Lottery.sol
Think for a second on what some of the functions that are going to be.What our main function going to be.
We'll probably have a function enter. function called getEntranceFee. function called startLottery. function called endLottery.
Let's get started with the enter function just because this is most likely going to be the entry point. As we know since we're going to want them to pay using this entry function in ethereum, we're going to need to make this function payable. In here we're going to need to keep track of all the different players.Everybody who signs up for this lottery.
address payable[]
To keep track of all the players we're going to make an address payable array.We'll make it public and called players.
And anytime somebody enters we'll just do:
require minimum payment
However we're not checking to see how much value that they're actually sending.We want to set the price of this to be atleast $50.Here we're gonna have to do a require statement requiring them to do at least $50.In order to do that we're probably going to need to have some function to get the entrance fee to check whether or not how much they're sending is actually $50.
getEntranceFee
Since we're just returning a number for getEntranceFee, we can make it a view and have it return uint256.
To get this entrance fee we're first going to have to have stored somewhere what the entrance fee is.We're going to store the $50 minimum somewhere.This is something we'd probably wanna set right when our contract is deployed.So where we can put stuff like that.Well in our constructor.
Since we're going to get a conversion rate, we're going to want to use a chainlink price feed.
We're going to need to pull from the price feed to convert fifty dollars to fifty dollars in eth.
We'll compile it and we can see everything works our properly.
So now we've pricefeed let's go ahead and set up this entrance fee.Of course we're going to need to get a price from the price feed.We can check the documentation on how to do that. We can call this latestRoundData function.
Now we're going to want to do a little bit of quick math.Typically if we're setting the price at $50 and we've a pricefeed of $2000 per eth, we'd just wanna do 50/2000 but ofcourse solidity doesn't work with decimals we can't actually just do this.So we'll have to do 50 * (somebignumber) / 2000.But first convert the price from int256 to uint256.
Since we know we're going to be using an ethereum / usd pricefeed that has eight decimals.Let's also just convert it to having 18 decimals as well.
uint256 adjustedPrice = uint256(price) * 10**10;
Now that we've adjusted price we'll do:
usdEntryFee is also multiplied by some big number.This way usdEntryFee has 18 decimals but it has an additional 18 decimals here that'll be canceled out with our pricefeeds.
Now ofcourse since we're doing some math here,it's recommended to use safe math and use safe math functions.We're going to skip over the safe math functions here again because in the newer version of the solidity you don't really have to use them but I think it's important to note here that sending this code this exact code into production would be a bad idea for atleast the reason of safe math functions.
Testing
Let's go ahead and do some testing as we code just to make sure our getEntranceFee function is working properly.
Based off of our last lesson let's talk about how do we want to test this?
Well we could do a mainnet-fork here because we're only working with some on-chain contracts and some maths.We'll at some point have to do our development with mocks and of course our test-net.
Let's try our mainnet fork just for now just to see if this is making any sense.The current price of ethereum is $3000 and we want price of this to be $50 we do 50/3000 =0.016666667.This should be approximately what we get for our eth value.
So if we were to test the function, we'd expect to get 0.016666667 or in wei 0.016666667 * 10**18.
Let's go ahead and create a function that tests this.
We need to import Lottery and in order to deplot it we need to get an account.We're going to import our helpful scripts from the last project to this one too so we can get our get_account function but for the time being we can actually just use accounts[0] from brownie accounts.
We've a parameter of _priceFeedAddress in the constructor of the contract.So for now let's hard code that in our config.And we can do our quick test.
These numbers are of course going to be a little bit different for you and if you want you can go ahead and skip the part so that you don't have to do the math but it's kind of nice to do quick sanity check saying based off what things are right now or what would this price end up to be.
In our last section we made a mainnet-fork-dev network.I'm going to go ahead and customize our mainnet fork the way we showed you guys how to do mainnet-fork-dev.
To do this we're first gonna have to delete brownie's internal built in mainnet-fork.
brownie networks delete mainnet-fork
and now add our own mainnet-fork using alchemy.
brownie networks add development mainnet-fork cmd=ganache-cli host=http://127.0.0.1 fork=https://eth-mainnet.alchemyapi.io/v2/YObK6vh1zJzmWL_IP2d_oIDI6tdl3ua5 accounts=10 mnemonic=brownie port=8545
Great mainnet-fork has been added.Now that we've this here we can go ahead and run our test.
brownie test --network mainnet-fork
costToEnter should be divided by adjusted price i.e uint256 costToEnter = (usdEntryFee * 10**18) / adjustedPrice;
Of course we know that we're gonna change assert statement because this isn't a great way to actually test our contracts but it can be a nice sanity check and we know we're gonna have to refactor the code as well for mocks and for accounts but we'll get to that in a little bit.
We're getting a cost to enter correctly.We can do in our enter function that value must be greater than getEntrancefee function.
We've a way for them to enter and we've a way to get the entrance fee.
Enum
But we wanna make sure that we're not ending the lottery before the lottery even starts or ending the lottery hasn't even begun.So we're gonna want a way to iterate through the different phases of this lottery and we can do that with what's called an enum.
According to solidity documentation, enums are another way to create user-defined types in solidity.We saw an earlier version of doing this with a struct.Enums are little bit different in that they're explicitly convertible to and from all integer types.So for our lottery contract we're gonna want to create the new type that represents the lottery state.
This means that we've a new type called LOTTERY_STATE with three positions.These different states are actually represented by numbers. So open is actually 0, closed is 1 and calculating winner is 2. Now that we have this new type we can create a variable of type LOTTERY_STATE.
Right when we initialize our contract, we're gonna want to set our lottery_state being Closed.
Since states are represented by numbers as well we could also just do lottery_state = 1; however it's much more readable to do LOTTERY_STATE.CLOSED.
Now that we've a lottery state, in our enter function we can require lottery to open.
We can only enter if somebody started this lottery and that's exactly what we're gonna do in our start lottery.
startLottery
Now when somebody starts the lottery they'll be able to enter.Of course the startLottery bid here need to be called only by our admin.So this is where our onlyOwner modifier is once again going to come into place.
We could write our own only owner modifier or we can once again use open zeppelin's access control and open zeppelin's ownable function instead which is what I'm going to use here.
and we'll say our lottery is ownable.
Now we can finally move into our endLottery function.This is where we're actually going to choose a random winner here.We only want the admin to be the one to end the lottery.so let's add the onlyOwner modifier in endLottery function as well.
Randomness
We're looking to get a random winner.As you know blockchain is a deterministic system and this is super ambitious because that allows us to do all these smart contracts and have this system that can actually reach consensus very easily.Random numbers are much harder.
If you've a blockchain with a whole bunch of different nodes and each nodes responds and gives their own random value, well each node is never going to be able to sync up and agree on a random number.
What you could do is base the random number on some other attributes in the system but it's not really random.It's actually going to be pseudo random.So getting truly random numbers in a deterministic system is actually impossible and if you know about computer science you actually know that even if you call math.random in your javascript or C++, what your computer is really doing is it's looking at some place in memory, grabbing some value and saying this is probably random enough here go ahead and use this.
Now in smart contracts especially when working with any type of financial application such as a lottery, having an exploitable randomness function means that your lottery is at risk of being hacked or destroyed.
So I'm going to show you this insecure way first and the reason that I'm gonna show you is that it's a quick and dirty way to get a pseud random number but please don't use this in any production use cases.
pseudorandom numbers
Im gonna show you a method that's often used to teach people how to get random numbers and then we're gonna explain why it's so vulnerable and not a good method of randomness and what some insecure protocols will do is they'll use globally available variable and hash it.So in your smart contracts there's actually number of globally available variables.One of those we saw above is msg.value.It's going to be the value that's sent with the transaction.Another globally available variable is going to be msg.sender.You can actually see a whole list of these different globally available variables in the solidity documentation here.
Since there are these globally available variables, alot of times some will see something like block.difficulty which returns the current block difficulty.Now one of these globally available variables is going to be block difficulty.Remember how I said the time between different block generation is called the block time.Well you can always keep that block time as is by changing the block difficulty over time.The harder the problem or harder the POW algorithm, the longer it's gonna take or more nodes you gonna need to solve that problem.There's this constantly recalculating metric called Ethereum Difficulty or block difficulty depending on the chain that you're working on that constantly changes.
You might think this would be a great use of randomness because it's a somewhat hard to predict number.So what alot of people do is they think that these sound pretty random and use them as a unit of randomness and you'll see something like.
We're converting into uint256.The reason that we're doing this is because we want to pick a random winner based off of an index.Somebody some random winner in our players array or our players list.Whatever number we're going to use that's gonna be the index of the winner that we're gonna randomly pick.Then we'll use Keccack256 which is our hashing algorithm.So they hash a whole bunch of variables together and they do this abi.encodePacked which is another keyword for some low level work and they'll add nonce, message.sender,block.difficulty and block.timestamp.Basically what they tryna do here is take a bunch of seemingly random numbers,mash them all together in a hashing function and say yeah this is pretty random.
But the issue here is that the hashing function itself here isn't random here.The hashing function is always going to be exactly the same Keccack256.It's always gonn hash everything exactly the same way.So we're not actually making it more random by hashing it.All these numbers inside are the pieces that actually determined how random it is.If the block.difficulty is random then this will be a random method.If the block.difficulty isn't random then this won't be a random method and block.difficulty isn't random.Block.difficulty can actually be manipulated by the miners.Timestamp, nonce and msg.sender are predictable.
When using a random number in this way the hashing algorithm is always gonna be the same, uint256 is always gonna be the same.We've predictable number, predictable address, a predictable timestamp and then a manipulatable value.So all the code is really doing is giving the miners the ability to win the lottery.So this isn't going to be an effective way to get a random number.This is an unacceptable way to get a random number in our applications.Yes we do have the onlyOwner modifier here which means that we're the ones who are going to choose when to call this.So it's still a centralized in that regard but let's just jump into best practices for working with random numbers.
True Randomness with Chainlink VRF
In order to get the true random number we're gonna have to look outside the blockchain.Blockchain itself like I said is a deterministic system.So we need a new number outside the blockhain but what we can't do is we can't use just an api that gives a random number.If that API becomes corrupted if they're malicious if they go down if something happens etc etc what we need is a provable way to get a random number and chainlink vrf is actually that solution.
Chainlink-verf stands for chain-link verifiably randomized function and it's a way to get a provably random number into your smart contract.It has an on chain contract that checks the response of a chainlink node to make sure the number is truly random.Using some cryptography magic it's able to check a number of the parameters that the chainlink vrf started and ended with to make sure that it's truly random.It's already used for protocols like avagochi, ethercards, pool together and a wholebunch of other protocols as well because it'a a secure, reliable and truly provable way to get a random number which is incredibly powerful in a decentralized system.So that's how we're actually gonna get our random number here.Let's work on getting that.
We can head over to chainlink documentation.IF you ever get lost or confused you can always come right back to here to work with it.What we're gonna do is deploy their simple version in remix to work with chainlink vrf.
You can see that this is actually using a different chain than what we've been using.We've been mostly working with rinkeby but for this demo It's actually on Kovan.Remember if you do want to stay with rinkeby you can go to contract addresses section of the vrf and grab the addresses.
So what's going on in this contract? How do we actually use it? Well first thing that happens is we're importing some code from the chainlink package and our contract is inheriting the abilities of this vrf consumer base contract.We can see what functions we're actually going to use that are inherited from this contract.And the first thing that we notice is we can see that our constructor in here does some weird stuff.Looks like it almost has two constructors.So what's actually going on here?Let's look at VRF ConsumerBase.sol contract in the chainlink github.As we can see the vrf consumer base that we're importing has it s own constructor.
It takes an address for the vrf coordinator this is the on chain contract that actually checks to make sure our numbers are random and the address of the chainlink token which we'll talk about ERC20s in a little bit.
What we're doing is also inheriting the constructor into our contract.So this is our constructor for our random number consumer but we can also use the constructor of the VRFConsumerBase and this is actually how we go ahead and do it.We grab the constructor of the vrf consumer base and pop it in.It's taking two addresses.It's taking vrf cordinator and the link token like I said vrf cordinator is a contract that's been deployed on chain that's going to verify that the return of the chainlink node is truly random and we're going to use the link token as the payment to the chainlink node fro it's services and then we also have a key hash and a fee defined inside of the constructor.As well as the key hash uniquely identifies the chainlink node that we're going to use and the fee is how much link we're actually going to pay to the chainlink node for delivering us this random number.
Oracle Gas and Transaction Gas
In ethereum whenever you make a transaction, you've to pay some eth gas or transaction gas.This is to pay smart contract platform a little bit of eth for performing our transaction with a smart contract.With a smart contract that operates with an oracle we've to pay some link gas or oracle gas.This is to pay the oracles a fee for their services for providing data or doing some type of external computation for a smart contract.
The question that might follow up is "Why didn't we pay oracle gas when working with chainlink price feeds?"
Well for price feeds somebody has actually already paid for the data to be returned and if we go to data.chain.link we can see list of sponsors that're paying to get this data delivered.They're already paying oracle gas to bring this data on chain for us.
Since no other protocol is getting a random number for us we're actually going to have to pay the oracle gas here.
Now in this RandomNumberConsumer.sol, we've a function called getRandomNumber which is gonna return a bytes 32 and what it's gonna do is call this request randomness function which is inherited from this VRFConsumerBase.If we look in here we look for request randomness, we can see there's a function right here called request randomness.This function is gonna send our oracle fee or the link token and it's gonna call this specific to the link token function called transferAndCall which is going to call a chainlink node.I'm not gonna go into exactly how it's doing that now but we'll talk about it in a little bit.So we call this request randomness function and we send the key hash and the fee.Remember the key hash uniquely identifies the chainlink node and the fee is going to be how much oracle gas we're going to pay.
If you ever get lost on how much to pay or what the contract addresses are, you can always head over to this vrf contract section and see where the most recently deployed vefs are, how much the fee is etc etc.
Request and Receive
Getting a random number is actually follows what's called the request and receive style of working with data.Let's go ahead and try this out and i'll explain what this means once we see it.We're saving the answer to randomResult variable here and let's just go ahead and try this and see what happens.
Kovan
We're gonna switch to injected web3 and since we're now swapping to a new test network this means that we've to get tested eth and test that link again.We can always look for the link token contracts page to find the most upto date faucets.
Now that we've some testnet ethereum and some testnet link we can proceed.We need to make sure we're on Kovan test network and we're gonna deploy our RandomNumberConsumer -gist.Metamsk pop's up and we're gonna confirm it.I didn't explined fulfillRandomness function intentionally you'll see why in a second.
randomResult is 0 at the begining because we haven't got a random number.
Gas Estimation Failed
I'm going to do something intentionally wrong because there's a good chance that you'll run into this at some point.If I hit getRandomNumber right now, we'll see the error Gas estimation failed.We've plenty of eth why would this fail? The reason that it's failing is because the contract doesn't have any oracle gas.So we got that gas estimation failed because we need to fund this contract address with some link to actually get a random number.So let's copy the contract address, come to the metamask and to that address we're gonna send some link.
After Link transaction completes
Now that this contract has some testnet link, we can call this getRandomNumber button because we can actually pay the chainlink node to actually return our random number.We're also paying a little bit of transaction gas to make this transaction to make the request and then we're paying a little bit of oracle gas to make the transaction.
After transaction completes
After the transaction confirmed but if I hit random result now it's still going to be zero.So why's that?What's going on?
Request and Receive
Getting a random number like this actually follows request and receive cycle of getting data.You can read more about it here.So in one transaction we actually request some data or in our case a random number and then in a second transaction the chainlink node itself will make a function call and return the data back to the smart contract.In this case the function that we're calling is fulfillRandomness.It calls fulfillRandomness with bytes32 requestId which is going to be the request number of when we call requestRandomness and it's going to return with random number called randomness.So after we wait a little bit and hit randomResult, we can see indeed our random number is in here.
Asynchronous 2 Transactions
Again the reason that it's in there because we actually had two transaction occur.One paid by us when we called getRandomNumber and one paid by the chainlink node when it called fulfillRandomness.
Clarification
Now I lied to you a little bit technically that VRF Cordinator contract calls fulfillRandomness function and then the chainlink node calls VRF Cordinator function but for simplicity's sake you can kind of think of it as the chainlink node itself is calling this fulfillRandomness function.
So now that we know how to do in Remix let's go ahead and add this to our brownie project.
endLottery
Before even get a random number let's change the state of our lottery.
While this is happening no other functions can be called.This will lock out.Nobody could start a lottery and nobody can enter a lottery while we're calculating the winner.Now that we know a little bit more about random numbers and everything that we're doing here.Let's go head and try implementing this.In our chainlink smartcontract docs, in our get a Random number section.First thing we're gonna do is import the VRFConsumerBase code and we'll inherit it into our Lottery contract.
If we scroll down to the constructor of VRFConsumerBase, we can see it address of the vrfCoordinator and the address of the chainlink token as parameters.We can use a constructor of a contract inherited in our contract inside our constructor.So what we'll want to do is we want to come down to our constructor and right after the public keyword we can add any aditional constructors from inherited smart contracts.
Similar to priceFeedAddress vrfCoordinator and link addresses are gonna change based on the blockchain that we're on.So it'll probably makes sense for us to parameterize them the same way we parameterize the pricefeed address.So in our top level constructor parameters we'll add an address for the vrfCoordinator and pass that to the constructor of the vrfConsumerBase.We'll also grab ab address for the link token and pass it to the vrfConsumerBase constructor as well.
Great what else we need to make this work?Well back in the documentation we can see:
We need fee and the keyHash.The fee is associated with the link token needed to pay for this request.So we'll make a public variable for fee.
Since fee might change blockchain to blockchain we'll have fee as an input parameter as well.
We need a keyHash.The key hash is a way to uniquely identify the chainlink vrf node.
Now that we've the main pieces that we need.How do we then request this random number? Well if we scroll down to ou documentation we can see we have this requestRandomness function that we need to call.If we scroll back to our vrfConsumerBase contract, this requestRandomness function is a built-in function from the vrfConsumerBase.So our contract can natively call this requestRandomness function right in our contract.You can see it takes keyHash and a fee as it's parameters.So right in our endLottery function we can add this requestRandomness function and we can also see requestRandomness returns requestId of bytes32.
returns(type variableName)
This return syntax is very powerful.You can actually identify the name of the variable you want to return right in your function declaration.So by saying bytes32 requestId we're saying we're going to return a bytes32 variable named requestId.
This function call follows again what we've talked about as the request and receive mentality.This means that in this first transaction we're going to request the data from the chainlink oracle.In a second callback transaction the chainlink node is going to return the data to this contract into another function called fulfillRandomness.
fulfillRandomness
Again if we look back in our vrfConsumerBase, we can see it has this function rawFulfillRandomness.We can read in comments that says "rawFulfillRandomness calls fulfillRandomness after validating the origin of the call".There's a little bit of contract tag going on but it's gonna eventually call fulfillRandomness function which is gonna be what we define in here and that's how our contract gonna know what to do once it gets the random number back.
So in our first transaction we're gonna end the lottery, request a random number and then a second transaction later on once the chainlink node has created a provably random number it's gonna call a second transaction itself based off of what we define.We just have to call it.
We don't want anyone else to be able to call this function.We only want our chainlink node to call this function so we can return a truly random number.So we made it an internal function.It's internal because actually the chainlink node is calling the vrfCoordinator and then the vrfCoordinator is calling our fulfillRandomness.So we made it internal so that only the vrfCoordinator can be the one to call and return this function.
override
This override keyword means that we're overriding the original declaration of the fulfill randomness function.Our vrfConsumerBase has a function fulfillRandomness defined but it doesn't have any parameters or anything about this function actually laid out.This function is meant to be overriden by us and that's exactly what we're doing here.
So in this fulfillRandomness function let's define what's gonna happen once we get random number back.Before we can process this random number let's just check to make sure we're even in the right state.So we'll do:
Then just do another check to make sure we actually get a response.
Now we need to pick a random winner specifically our of our list of players.List of payable public players.So our players array is just a list of players.
Modulo(Mod Operation %)
What we can do to pick a random winner?We can do a modulo function.We can use mod function in our fulfillRandomness with the length of our players.
And just to keep track let's make a new variable called recent winner.
Now that we got a winner wallet we want to pay them all of the money gathered from our enters.We'll do is transafer the entire balance of the address.
We'll transfer them everything we've.Then of course we're gonna want to reset the lottery so that we can start from scratch.We can start blank again.So we'll do:
We're gonna change our lottery state to being closed because the lottery is now complete.And I often also like to keep track of the most recent random number.So at the top we'll declare public randomness.
Some of the changes I need to fix to compile properly is here:
I need to have v0.6 instead of 0.8 in my code while importing VRFConsumerBase.sol.
import "@chainlink/contracts/src/v0.6/VRFConsumerBase.sol";
also need to make recentWinner variable payable.
address payable public recentWinner;
and LOTTERY_STATE state to be CALCULATING_WINNER instead of CALCULATING_WINNERS.
Let's now move into our testing and developing phase.As you're starting to figure out we can ignore these file import callback not supported bugs in vscode for now.I'm sure solidity linter will get better as time progresses.
Lottery Testing
Let's go ahead into our scripts and we'll make a deploy script first.
Our default network is not defined.So in our brownie-config.yaml:
If we run this brownie run scripts/deploy_lottery.py, ganache will spin up and nothing will happen because our deploy_lottery function doesn't do anything right now.
deploy_lottery.py
First thing we need always to deploy a contract is we need an account.
Files inside our scripts directory are:
And this is what get_account function looks like in our last project.
Along with the function we also had imports and vriables.
get_account refactored
Let's flush this out just a little bit more so it's even more robust.Right now we've a way to use brownie's ganache accounts and use our environment variables.However there's a third method that we learned that isn't identified here is accounts.load("id").If you still have your account and you still have all your brownies set, up you could do:
brownie accounts list
You could see couple accounts or atleast one.We want our get_account to be even more liberal so that if you wanted to use one of the listed account, we could.Let's modify our get_account script.
First get rid of else and that indent.
What this will do is that this getting from config will be our default.IF nothing that we define prior to this, we'll just default to grab from the config.Since we're doing that let's create our .env file.We'll paste our privatekey, infura and etherscan token here.And in our brownie-config ofcourse we'll:
Let's add index and id.
This way if we pass an index to our get_account function, we'll just use index from accounts variable and if we pass an id, we'll have try to do accounts.load.Let's change a little bit.
Now we've much more liberal get_account function.So now we've a account we can actually deploy our lottery.
We're gonna have to add in all these different variables so if we go back to our lottery we can see :
We need to give it a _priceFeedAddress, _vrfCoordinator, _link, _fee and a _keyhash.Now the way we did that in fund me is that we did it in way where we check to see if we are on a local chain or not.If we weren't on a local chain then we must pull our addresses directly from our config.If we weren't on a local chain though we'd deploy some mocks and use the address of those mocks.
We're gonna wanna do the same thing here but let's make our lives little bit easier.We can wrap all of the mocking and checking into a single function get_contract which we'll add in helpful_scripts and import it to deploy_lottery.
If this is little confusing to you we'll go over to chainlink mix in a little bit and inside of this has a more robust description of what's going on this get_contract.You can check that out if you want to learn more.
So we obviously want contract name as a parameter.Meaning that this get_contract function will get something like:
get_contract("eth_usd_price_feed")
That's gonna be the same as what's defined in our brownie-config.yaml file.From this name we're gonna get the type of contract that it is.To do that we're gonna have to create some mapping that'll map the contract names to their type.
contract_to_mock
eth_usd_price_feed is gonna be of type MockV3Aggregator and also we need to import MockV3Aggregator.
Now we need to check do we actually need to deploy a mock.Let's check if we're on a local blockchain.And we'll skip the forked local environments because again we don't need to deploy a mock pricefeedAddress on a forked local environment.Also we'll check if one of the contracts in contract_type has been deployed or not.
if len(contract_type)<=0 is equivlent to MockV3Aggregator.length
We're checking how many MockV3Aggregators have actually been deployed.If none of them have been deployed, we'll be deploying them.
We're gonna have to create this deploy_mocks function which is going to be same as what we did in our brownie FundMe.
Now we have a way to actually deploy the mock pricefeed.Let's go back to our get_contract function.So we have a way to deploy the mock if one isn't deployed.Now what we're gonna want to do is we want to get that contract.WE want to get that mock.
This is saying let's grab the most recent deployment of the MockV3Aggregator which is exactly what we want.This will work perfectly for our development context.However we're not always going to just want to deploy to a development network.We're also going to want to deploy to testnets.
For example if it's eth_use_price_feed, contract_name will be eth_usd_price_feed and the way we're setting it up it's got to be the same as what's in our contract_to_mock dictionary.Inside If the way we did it was we actually got the contract because we had it's contract type based off of brownie.Here we've to interact with the contract getting those two pieces that we always need:
- Address
- ABI
We actually have the ABI from our MockV3Aggregator type and we just got the address.So we can create the new contract type.
Contract.from_abi
This contract package can be imported from brownie and it has this function from_abi that allows us to get a contract from it's abi and it's address.
We gave it a name(contract_type._name)
MockV3Aggregators and all these contracts have a .abi attribute that returns the abi. They also have ._name which returns their name.
This is how we'll get the contract otherwise.And then at the end of all this we'll just return contract.
This is an incredibly powerful function for us to get a contract based off of if it's already deployed as a mock or it's a real true contract.
Now that we've this function let's go back to our deploy_lottery.
Let's go this again.This Lottery.deploy get_contract is gonna get an eth_usd_price_feed.If we don't have a mock deployed, it's gonna deploy a mock pricefeed for us and we're goint to return that mock pricefeed.However if we're on a testnet or a real network, we're gonna grab it's actual address and return a mock contract of it.Our mock contract has all the same functions of a regular contract.So we can just use it as the same.This way we don't have to adjust deploy_lottery function for whether or not we're deploying to a testnet or to an actual address.
The only additional piece we should put in here just for clarity is we should add .address because get_contract is gonna return the actual contract and we really only want the address.
This is going to make our coding alot more robust for moving between development environments and testnet environments and working with scripts in a really effective manner.
Let's go back to lottery and figure out what are the different pieces we need.
We also need a _vrfCoordinator.
So to do this we're gonna go back to docs.chain.link, then to contract addresses and we could do mainnet here but let's also just set this up to work with rinkeby.We'll copy the address of Rinkeby's vrfCoordinator and add it to a new network in config file and while we're doing this let's also get the rinkeby address for our pricefeed, grab the address for eth/usd and add it in a config file.
Adding vrfCoordinatorMock
Now we can do to get vrfCoordinator contract address.
Ofcourse we're gonna have to go back to our helpful scripts because there's currently no mapping between what a vrf_coordinator is and what it's mock needs to be.We need to get a mock vrf_coordinator.First of all we don't even have our MockV3Aggregator.So let's go grab that as well.We can grab both of these at chainlink-mix.Let's go to VRFCoordinatorV2Mock and grab whole thing and create new directory "test" inside contracts and inside test create "VRFCoordinatorMock.sol" file and past the code there.Also create file for MockV3Aggregator and past the code.
VRFCoordinatorMock has different functions for actually working with the vrf_coordintor.One of the specific ones in particular is this callback with randomness that we're going to use in our tests.So our vrf_coordinator is also going to get mapped to our VRFCoordinatorMock and we'll also import it from brownie.
Just to double check everything's working we wanna run quick brownie compile to make sure we're importing our mocks correctly and everything is compiling.
We need a link token now.The chainlink token is just another smart contract.So we're gonna do the exact same thing here.
This means in our config for rinkeby let's add a link token address.Go to chainlink documentation and to link token contract, grab the address and put it in config file.
Now we've it in our brownie-config.We also need to add it to our helpful_scripts.We need a mock link token for that which we can get from here.We can just go ahead and grab everything, create a new file "LinkToken.sol" inside test folder and past it inside the file.In our helpful_scripts we're gonna map link_token to LinkToken and import LinkToken from brownie.
We need a _fee and a _keyHash.The _fee and the _keyHash are both just numbers.These aren't actually contracts.We don't need to put this through get_contract bit.What we can do is just in our config, in our development network, we just add a default for the keyHash and fee.I'm just gonna set my development keyhash and fee equal to the rinkeby ones.You can grab these from here.
In our deploy_lottery we can just grab this directly from a brownie config because we're always gonna have the default keyhash and fee.
Then ofcourse our last bit here.
Then additionally as we learned last time if we want to publish this :
What this is saying is get that verify key but if there's no verify key there just default to false.This way if we don't set a verify key in development, that's fine it won't get verified.For rinkeby set verify to true.So we can actually verify this on a rinkeby chain.
Alright let's try this out.We'll do :
brownie run scripts/deploy_lottery.py for default development network
We run into this error.
We get an issue with our get_contract function.We forgot to add deploying the mocks in our deploy_mocks function.Right now our deploy_mocks only deploys the MockV3Aggregator.Let's add the rest of the mocks in here.We're gonna need to deploy VRFCoordinatorMock and LinkToken.So let's open up those contracts see what they need.LinkToken doesn't have a constructor.We can just have it to be blank.
Let's see what that VRFCoordinatorMock takes.
It takes the link token as an address.
Let's try again deploying it again.
We can see that our MockV3Aggregator was deployed then our LinkToken was deployed then our VRFCoordinatorMock was deployed and then our lottery was deployed with those mocks defined.We deployed the lottery.
We could 100% go ahead and then run this script on an actual testnet because our config is set up well.Let's write more functionality for actually interacting with this lottery before we actually do that and then we can actually just run a script which will do all this functionality end to end because again deploying to testnet takes long time and we really only want to do that when we're done and fairly confident that everything's working well.
Python Lottery Scripts/Functions
What's the next thing that we'd want to do?Well we probably want to go ahead and start the lottery.Let's write a script that can actually do that.
Start Lottery
We said that the lottery is gonna be the most recent deployment of the lottery and we're gonna call startLottery function of Lottery.sol.It is indeed changing state.So we do have to make a transaction.Then in our main function we call the start_lottery function.So if we run this again on our development chain.
We did run into an issue and this is something you'll see something from time to time.
Typically workaround is you wanna wait for that last transaction to actually go through.So we'll say:
enter_lottery
Next we wanna do is enter the lottery.We need to pick some value to send when we call the enter function because we need to send the entranceFee with it and just to be safe I usually will tack on a little bit of wei as well because sometimes it might be off by like 1 or 2.
end_lottery
Before we actually end the lottery, we're gonna need some link token in this contract because our endLottery function calls the requestRandomness function.We can only requestRandomness if our contract has some chainlink token associated with it.So we're gonna need to first fund the contract and then end the lottery.
Funding with LINK
Since funding our contracts with the link token is gonna be a pretty common function that we use.Let's go ahead and turn this also into a helpful_scripts.
We have a contract address then we wanna who we're gonn fund with link.We'll set default account to be none.We'll do same with the link token as well.If you want to use a specific link token you can otherwise we'll just grab it ourselves and then we'll also do a default amount which will set to 0.1 LINK.
First we get an account.We'll be doing a little clever python stuff.We'll say account = account if somebody sent it otherwise we'll call our get_account function.Then we'll do the same thing with the link token.Now that we've the link_token contract from our get_contract function which again is basically the equivalent of Contract.from_abi on our link token, pulling from our config or from our mock.Now we can just call functions on the link_token.We're gonna transfer token to contract_address with a certain amount and from account.
brownie interfaces
I do however wanna show you another way to work with this.Using the interfaces.Instead of doing the link_token.transfer directly on the contract, we can use the interfaces section to actually interact with some contracts.Right now we've our LinkToken inside contracts/test directory and it has all the definations and all the functionalities defined.Sometimes you're gonna have contracts that you don't have everything.You don't have all the functionality and maybe only have the interface or some functions definitions.So we can still interact with contracts with just an interface because again that interface will compile down to our ABI.
So as a another way of teaching us how to actually work with some of these contracts, what we can do is we can use the link token interface basically the same way as we use the link token contract here.This will compile down to a way that our brownie package knows how to interact with these contracts.If you wanted to, you can just grab it, go to our brownie sections and inside interfaces, create a new file "LinkTokenInterface.sol" and paste the code.
What we can do in our helpful_scripts instead is first import interface from brownie and pass contract address.
This is an another way we can actually create contracts to actually interact with them.So saw up in the code that Contract.from_abi which is great and interface is another way we can do the exact same thing.
If we've the abi, we can just pop it into Contract.from_abi.If we've the interface we don't even need to compile down to the abi ourselves because brownie is smart enough to know that it can compile down to the abi itself.
For now I'm gonna comment these two lines though and we'll just use the link_token.transfer for now.
Now that we've a funding function we can import fund_with_link from our helpful_scripts.
We're only passing contract_address in a fund_with_link functions because the way we've set the function will just automatically grab a default otherwise.And Once we were funded with link, we called endLottery function because again this is going to call that requestRandomness function from the chainlink vrf.
waiting for callback
From our end this is really all that we need to do but remember when we call the endLottery function, we're gonna make a request to a chainlink node and that chainlink node is going to respond by calling the fulfillRandomness function.So we actually have to wait for that chainlink node to finish.Typically it's within a few blocks.So normally what we can do is sleep for some time and we also need to import time.
We can see who that recent winner is if that chainlink node responded with a recent winner.
You might be thinking "There's no chainlink node watching our local ganache" and you're exactly correct.
What happens when we add our end_lottery function to our main function? Do you think that we're actually gonna get a recent winner back?Let's give it a shot.
brownie run scripts/deploy_lottery.py
While we're waiting here, this is when a chainlink node would go ahead and stat responding with our randomness.However as you probably astutely telling there's no chainlink node that's going to call this fulfillRandomness function right now.So for our ganache chain this will hypothetically end with nothing because there's no chainlink node actually responding.
For our testing purposes we're gonna figure it out how to actually get around that end the deal with that.We want to make sure that our tests are really solid on a development chain before we actually test this on an actual testnet.So let's jump into some of these tests.
We've already started with test_lottery.py but we're gonna iterate on this and make this even better.Now before we get into these tests there's a couple of things we wanna talk about.
Integration Tests Vs Unit Tests
Unit tests is a way of testing the smallest pieces of code in an isolated instance.We're going to use it loosely define testing independent functions in our lottery contract.
We also want to do integration testing which is gonna be testing across multiple complex pieces.Typically I like to run my unit tests exclusively on a development environment and my integration tests on a testnet.This is really helpful because we can test the majority of our application like we said on a development network and then still be able to see what actually happens on a real testnet and etherscan.
Typically what people do is in their test folder they'll create two different folders.One for unit and another for integration.Since for this demo we're only going to have one file for both.I'll not create those folders but it's pretty common practice.Instead I'm gonna rename the test file "test_lottery.py" to "test_lottery_unit.py" for unit tests and we're gonna create a new file called "test_lottery_integration.py".
Unit Testing
Now when writing the unit tests we really wanna test hypothetically every single line of code in our smart contract.This is incredibly important of course because smart contracts are open to everybody to see and interact with.So we really want to test every single line of code we've in Lottery.sol file.
Let's go ahead and finish writing a getEntranceFee test that will work on a local development network.As you can see we're already gonna refactor our test_get_entrance_fee function.So delete everything inside that function.
First we need to deploy our lottery.Since we've a deploy_lottery script already, we can just use the deploy_lottery script as well.If we wanted to we could just copy paste the whole code inside deploy_lottery function into our test but we'll work from the deploy_lottery script.
Now we import the deploy_lottery function.
And in our unit test we'll say:
lottery = deploy_lottery() which will give us our lottery.
Once we've our lottery contract, we can just call getEntranceFee.
entrance_fee = lottery.getEntranceFee()
And we wanna make sure the entrance_fee is what we expected it to be.So what we expect it to be? Well again in our helpful_scripts is going to deploy mocks and the initial_value is 2000.So if the price of eth is 2000 eth/usd, and the usd entry fee is 50, we'd say 2000/1 == 50/x == 0.025.We can go ahead and even do the math in the script.
And to test it we'll do brownie test -k test_get_entrance_fee
This is working exactly as we anticipated.
pytest.skip
As we mentioned since this is a unit test, we really only want to run this when we're working on a local blockchain environment or local development network.So we'll go ahead and do this with pytest again.
If you try to run this brownie test -k test_get_entrance_fee --network rinkeby, it should skip the test.
test_cant_enter_unless_started
Enter if going to be the first thing that these developers do.We can be more specific than that.We don't want people to enter our lotteries unless the lottery is actually started.So we make sure require (lottery_state == LOTTERY_STATE.OPEN) of enter function actually works.
Now when people try to enter a lottery that hasn't started yet it's gonna revert.
And ofcourse we gonna have to import get_account from scripts.helpful_scripts and exceptions from brownie.
Let's go ahead and test this brownie test -k cant_enter_unless_started.That'll pass as well.
test_can_start_and_enter_lottery
We've tested whether or not they can't enter.Let's test whether or not they can enter.
We asserted to check that we've correctly added a player to the lottery because we've our players array and we're asserting that pushing them onto our array correctly.Let's try this now brownie test -k test_get_entrance_fee.
test_can_end_lottery
Let's test to see if we can actually end a lottery.
Now we actually end the lottery we do need to send the link because we're calling requestRandomness.We use our fund_with_link function that we've in our helpful_scripts.We'll call fund_with_link on our lottery contract.Once we fund with link we'll then call endLottery.
Then how do we actually know that endLottery is being called correctly?Well if we look back in our lottery contract.When we call endLottery, we're not doing a whole lot.All we're doing is changing our state.So let's go ahead and check to see if our calculating_winner state is different.
Calculating_winner is in position 2.
test_can_pick_winner_correctly
Now let's test the most interesting piece of this entire lottery contract.We're gonna test whether or not our fullfill function actually works correctly.Does fulfillRandomness function of Lottery correctly choose a winner?Does it correctly pay the winner?Does it correctly reset?
Let's go ahead and build our most complicated and most important test of this whole contract.
These are going to be different accounts because we want to test for multiple different people.This unit test is getting drastically close to being an integration test but as I said we're being a little bit loose with the definitions.Now we're gonna want to fund it with link and now we're gonna choose a winner here.This is where we actually have to modify one more thing in our lottery contract.
Events and Logs
In order to test actually calling this fulfillRandomness function and testing everything in here, we're going to need to call this fulfillRandomness function.If we look in our VRFCoordinatorMock, we've this function called callback with randomness and this is the function that actually calls this rawFulfillRandomness.selector which eventually will call that fulfillRandomness function but this is the entry point that the node actually calls.We've to pretend to be a chainlink node and call callBackWithRandomness function.We're gonna return a random number, choose the contract we want to return to but we also have to pass the original request_id associated with the original call.
Now in our lottery contract, our endLottery function isn't gonna return anything and even if it did, it'd be really difficult for us to get that return type in our Python.So what we wanna do is to keep track of when this contract actually entered the calculating_winner state is we want to do what's called emitting an event.
Events are pieces of data executed in the blockchain and stored in the blockchain but are not accessible by any smart contracts.Events are much more gas efficient than using a storage variable.You can kind of think of them as the printlines of blockchain.
You can go to the logs section in etherscan which also includes all the different events.We're actually going to do an event ourself just so that we can see what this really looks like.
When we called endLottery function, in the logs if we scroll to the bottom, there's an event here called RandomnessRequest which was spit out by the vrfConsumerBase that we inherited and it even has some data that's already been decoded.One of those pieces of data is the request_id.
Now to add an event we first need to create our event type.
Now we've identified a new type of event called RequestedRandomness.It's really similar to enum in this regard.To omit one of these events, all we've to do in our endLottery bit is we'll emit RequestedRandomness.
Because RequestedRandomness takes a bytes32 as an input parameter and we're gonna pass it that requestID as an input parameter.Now that we've this event being emitted, back in our test when we call endLotter, it'll actually emit one of these events to our transaction.
Txn has a attributes called events which stores all of our events.We can then looked for a certain event name which we know is RequestedRandomness and in there in that RequestedRandomness event find the requestId.
These events are really helpful for writing tests, upgrading our smart contracts or understanding when a mapping is updated.
Now that we've this request_id we can pretend to be the chainlink node and use this callBackwithRandomness function to dummy getting a random number back from the chainlink node.So we're gonna call our get_contract function to get that vrf_coordinator and we're gonna callback with callBackWithRandomness function.
callBackWithRandomness
We need to pass the request_id of random number and then the contract to return to.STATIC_RGN is some random number which is the random number that we're gonna return and lottery.address because we're gonna return it to the lottery.From account because it's making a state change.
This is us dummying getting a response from a chainlink node and this is how we mock responses in our tests to make our lives way way easier.
Now that we've got our callback we can do our asserts.We can figure our who the winner of the lottery actually is.There are 3 entries that means it's 777 % 3 = 0 i.e our account is gonna be the winner.
We transfer them some money
Test this function out.
So go ahead and do our last bit.Create our integration test which we'll run on an actual chain and then if we wanted to we go ahead and try our deploy_lottery function on a real chain as well.
Integration Test
This is where we're actually going to test on a real live chain.We're gonna use rinkeby because that's the network that we've been using this whole time.Now I'm being a little bit liberal in the way that we're doing our tests here and we're just gonna do one test for this integration test but keep in mind you're gonna test every piece of your code.So let's just create a test.
This is going to be opposite of our unit test.Our unit tests only going to be on our local blockchains.We're gonna skip if it's not on our local blockchains.We're gonna do the opposite of pytest.skip thing.
Rest will be similar like we did in unit test.
account is twice cause we want two people to enter the lottery.
After endLottery is where this is gonna be a little bit different from our unit tests.In our unit tests we pretended that we're the vrfCoordinator and we called the callback with randomness.We pretended that we're a chainlink node.But here we're not a chainlink node because we're on a actual network.So we're just going to wait for that chainlink node to respond.So for simplicity we'll just do time.sleep(60) for it to respond.And since we're using time, we need to import it.And then since account was the only one to actually be in this.so we'll do assert for it.Also we'll do assert for balance.
we've an integration test which is going to run through pretty much vast majority of our functionality here.We can go ahead and test this with brownie test -k test_can_pick_winner --network rinkeby -s and before we run this we need to make sure we've test ethereum and link.-s flag will print out whatever brownie is going to print our.
Test deployment
Now that we've added our test, we can run our entire test.
for development brownie test
for testnet brownie run scripts/deploy_lottery.py --network rinkeby
First we go ahead and deployed our lottery.We got our addresses with our get_contract method, got our fee, keyhash, published_source and everything.Then we went ahead and verified it because it has this published_source.So if we grab the address "lottery deployed address", jump over to rinkeby etherscan, past the address in there we'll see the contract is checked mark.If we go to the read contract, we can see all the public variables and all public functions.We can go to write contract and see all the transacting functions that we can interact with.
After we get verified and things get deployed we went ahead and called our startLottery function to actually start the lottery then we entered lottery.We then funded the contract with linkSo we can get our random winner back.Right now our endLottery is confirmed and we're just waiting this 60 seconds because we did time.sleep.We can see on the transaction of that deployed lottery address in rinkeby etherscan, you can see different methods calls.
We can also go to events and we can see some events that we created.
We can see the first event is RequestedRandomness event.This is the event that we called.We can also see an OwnershipTransferred function that got called when we actually deployed this in the first place.
Topics
So topic 0 hash represents entire event.Hash near Hex is our topic 1 which represents that requestId.That is the requestId.
Conftest.py
One additional piece that I wanna talk about again in our testing a file that you often gonna see is "conftest.py".Python automatically knows to look for this conftest file and we'll grab different functions from it.We can also add external plugins, fixtures, hooks and test root paths.It adds alot of fantastic features and is a common thing that you'll see.We skipped here for simplicity but in future projects you'll probably see this conftest file which has alot of really nice testing configuration pieces.
That was a lot of stuff to code and quite frankly I don't think any of us want to have to do that every single time have to code everything from scratch.There's actually even easier way first to start with a blank project here and this is with brownie mixes.It has ton of boilerplate code for us to go ahead and get started to start start developing.The one we're gonna be working with is chainlink-mix piece.This gives us some wonderful contracts, tests, brownie config and really everything we need to do to get started.If you've brownie installed, you can just bake this mix.
brownie bake chainlink-mix
We'll get new chainlink folder with everything inside of it.In contracts we've a whole bunnch of different samples of working with the vrf, pricefeed, keepers to automate our smart contracts and making api calls and delivering any api call that we want to the blockchain.It also has a brownie config which already has a number of wonderful pieces in each networks.So that we don't have to go copy paste and add it in here.It's even got support for testnets like avalanche, polygon, binance and more.It has a number of really powerful tests including testing some pricefeeds.It has whole bunch of deployment scripts, some mocking scripts, some helpful scripts and really everything that we need to get started and get going.For starters we can run brownie test and it's going to compile everything and then on a development chain run all the unit tests and if we want to test on a real testnet, we could do brownie test --network rinkeby
If you're looking for a good starter place that has alot of really powerful smart contracts for you to get started, I highly recommend using this mix as a boilerplate starting point for any of your contracts or any of your projects.
We've learned a ton of fundamentals of working with brownie and smart contracts.Now that we've kind of all the building blocks though we can actually code alot of these much faster and much more efficiently than before.
One of the things that we've seen over and over again is working with tokens and the ERC-20 token standard.We're going to learn how to build our own token.Before we do that let's understand why we'd even want to do this.
ERC20s are tokens that are deployed on a chain using what's called the ERC20 token standard.You can read more about ERC-20 token standard here.Well basically it's a smart contract that actually represents a token.So it's a token with a smart contract.It's both.Tether, chainlink, uni token and dai are all examples of ERC-20s.Technically chainlink is in the ERC677 as there as upgrades to the ERC20 that some tokens take that are still backwards compatible with ERC20s.So basically you can think of them as a ERC20s with a little additional functionality.
Why make an ERC20?
- Governance Tokens
- Secure an underlying network
- Create a synthetic asset
How do we build one of these tokens?
All we have to do is build a smart contract that follows the token standard.Build the smart contract that has functions like name, symbol, decimals etc as mentioned in here.We need to be able to transfer it, get balance of it etc.Again if you want to check out some of the improvements that are still ERC20 compatible like the ERC677 or the ERC777, you can definitely check those out and build one of those instead.Let's create our new folder and get started.To create new brownie file:
brownie init
It's going to startup our repository and everything that we're going to work with here.We can kind of actually just jump in and start with our contracts.Let's create a new file inside contracts directory called "OurToken.sol" and this is where we're gonna add all of our token code.
Since this is an EIP, we could grab all the functions and past them inside our file "OurToken.sol" and then you know code all these up like these does some stuff blah blah.Or we could do it the much easier way.Since we're engineers we don't always want to reinvent the wheel.So once again our friend OpenZeppelin have some amazing contract packages for building our own ERC20 token.
This is the entirety of the code that is needed to make an ERC20.We can copy and paste this.Boom we've our ERC20 token.Let's actually change the name of everything here though.We're gonna be using solidity version 0.8 which I know I've done alot of this in 0.6 but I highly recommend working with 0.8 because it has a lot of really fantastic improvements to solidity.
Solidity 0.8
You'll notice this is our first dive into using a different version of solidity.You'll notice that most of the syntax is exactly the same.There are like I said couple of nice improvements with 0.8.The main one being you no longer have to use those safeMath functions that we talked about before.
Then we go ahead and import openzeppelin.Since we're importing openzeppelin, we're gonna add this to our brownie config and before we even get started, you technically have all the tools that you need to code and deploy your own ERC20 token.
Now I'm actually gonna challenge you to go ahead and try to start a brownie project, create your own token using openzeppelin packages and then deploy it on a testnet.I'm going to show you how to do all of it anyways but challenging yourself and trying to do things your own way and exploring is really one of the fastest ways to learn and grow in this space.
We need to add dependencies in our brownie-config file.
If you go to Github of OpenZeppelin, you can see the latest release.4.5.0 is the latest relaease at this time.So I used it.
Now we can import ERC20.sol contract from OpenZeppelin.Let's change the contract name to OurToken which we're going to inherit from ERC20 from OpenZeppelin.In our constructor, we're going to add our initial supply which is going to be in a wei and we're going to use the constructor of ERC20.It uses name and a symbol.We'll give a name to our token as "OurToken" and the symbol we'll do "OT".This is literally all we need for our token here.
1_deploy_token.py
For scripts all we have to do is create a new file "1_deploy_token.py" and "helpful_scripts.py".We'll quickly add get_account function in our helpful_scripts.You can go ahead and just copy paste from the last one.And to make sure add init.py inside scripts folder too.
Now in our deployment token script,we'll do our imports.
Since we knoe that we need an initial supply in our "OurToken" contract, let's go ahead and go that.
If you don't have your environment variable set, add your .env file where you can add private key, web3 infura project id and ether scan api.And then last but not least we'll add .env in our config.If you want to actually deploy this to testnet, you need to make brownie know where to grab private key from.
We can run the scripts
brownie run scripts/1_deploy_token.py and you will see our token is printed out.
We can see we've our token deployed on our local ganache.I can run this again on rinkeby.
Grab the address and go to rinkeby etherscan, pop it in and after quick refresh, You can see contract been added.You can even go to your metamask then to assets, add the address to our metamask, add the token and you'll see you'll be the owners of "OurToken".
Don't do this now
Something else you might wanna do is add this to a liquidity pool or add this to a place where you can actually go ahead and sell it and put it on the market.You can do something like that as easily as just popping onto uniswap, going to pool, hitting more, creating a pool and then adding our token.We'd have to manage the token list and be sure to add the token, create our own pool and automatically put it on uniswap.That's how easy it is to actually sell it on one of these pools.
I thought we know a little bit more about Defi and why it's such an amazing technology that only works in the blockchain world.Let's look at this site defi pulse or if you want to look at another one called defilamma which also shows alot of different protocols.So defipulse is an application that shows some of the top defi projects based on how much total asset is locked into each protocol.Aave which currently is the number one ranked defi application with 9.32 billion in assets under management in terms of USD.We've some fantastic ones like curve, compound, maker, InstaDApp etc.Now we're gonna be looking at two of these protocols in particular.The first one ofcourse being Aave and second one is gonna be what's called a decentralized exchange "paraswap".You start with aava.We're going to go to testnet.aave.com that looks something like this with please connect your wallet and everything will be pretty empty.
Paraswap is what's known as a dex or a decentralized exchange.It allows us to trade tokens incredibly easy on the blockchain.Now there's not a whole lot of testnet indexes that actually work.So we're just going to look at them and simulate as if we're working on them.For example if I wanted to trade 1 eth for some usdt or maybe some wrapped bitcoin or some dai or some aave or really any token that we wanted, all we need to do is connect to our wallet and swap button would show up and we could go ahead and hit swap.Some of these really powerful these dexes and really popular ones are going to be curve finance, uniswap and it's really really easy way to go ahead and swap your tokens for one another depending on what you're looking to do.
Aave is incredibly powerful and it's going to be one that we're going to be working alot with because it has pretty much all the fundamentals of working with the defi protocol that we're looking for and it has a testnet that we can go ahead and test and simulate actually working with some of these protocols.So we're gonna first work with the UI or the user interface and then we're going to do everything here programmatically.
Get some Kovan ETH
First things first if you don't already have it make sure you've some testnet ethereum at least.You can get some testnet ethereum from looking at the link token contracts and going to Kovan.Once we see some eth in our Kovan testnet we can go back to aave.So whereas paraswap allows you to simply swap between assets and do a lot of buying and selling of tokens, aave is a lending and borrowing application.So we can actually put down a token as collateral and we can borrow and we can generate some yeild from interacting with this protocol.Borrowing and lending is a critical piece to doing any type of really interesting financial applications or financial instruments such as short selling, being exposed to more assests etc.You can also gain some percentage back.
If you look at apy, it'll tell you how much percentage over a year you'll actually get in returns from staking or depsiting an asset.We're gonna connect the application using metamask and you'll see on a asset to supply, we can see our balance.
Depositing tokens/ Lending
What we can now do is click on ethereum, click on max button to deposit some ethereum, hit continue and a little dashboard will pop up asking us "would you like to deposit?".Clicking the deposit button will actually have us deposit right onto the aave contract on the Kovan testnet.Make sure once you hit the deposit that you're actually on a tetnet.
I did it on a rinkeby testnet.Idk why my Kovan isn't working while doing this.
