-
Notifications
You must be signed in to change notification settings - Fork 6
PAPI + IPFS script #103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
PAPI + IPFS script #103
Changes from 9 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
7f40aeb
papi script WIP
x3c41a b1cc189
WIP: seems like storing content with Smoldot and PAPI works, but gett…
x3c41a 74b1c9e
papi now works with ipfs
x3c41a b83f43a
consolidated package.json
x3c41a f6b5506
made .papi folders generatable with script, added gitignore, updated …
x3c41a b6cef81
reverted timeout
x3c41a 1145f03
updated README
x3c41a b5c767c
updated README
x3c41a 5cc2d6c
merged with main
x3c41a 9246d9c
Apply suggestions from code review
bkontur 33602fa
Update scripts/ipfs-reconnect-solo.sh
bkontur b99b605
Update examples/authorize_and_store_papi.js
bkontur 2eaefd8
Update README.md
bkontur b3ec7b0
Apply suggestions from code review
bkontur 7a7261d
Apply suggestions from code review
bkontur 4bb8d10
Update examples/authorize_and_store_papi.js
bkontur 994af7c
Update README.md
bkontur 64f3f4e
Merge branch 'main' into auth_papi
bkontur a46f4a5
Some nits - move package.json to the examples/
bkontur 805a16a
Let's not wait for finalized block
bkontur f7605c3
Nit
bkontur 55992cb
Update examples/README.md
bkontur File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # Using PAPI with Polkadot Bulletin Chain | ||
|
|
||
| ## Setup | ||
|
|
||
| 1. **Install dependencies (from the root of the repository):** | ||
| ```bash | ||
| npm install | ||
| ``` | ||
|
|
||
| 2. **Generate PAPI descriptors:** | ||
|
|
||
| Make sure your Bulletin node is running, then generate the type-safe descriptors: | ||
|
|
||
| ```bash | ||
| # From the root of the repository | ||
| npm run papi:generate | ||
| ``` | ||
|
|
||
| This will: | ||
| - Connect to your local bulletin chain at ws://localhost:10000 | ||
| - Download the metadata | ||
| - Generate TypeScript types in `.papi/descriptors/` | ||
| - Create metadata files in `.papi/metadata/bulletin.scale` | ||
|
|
||
| > **Note:** The `.papi/` folder is generated and should not be committed to git. It's listed in `.gitignore`. | ||
|
|
||
| 3. **Run the example:** | ||
| ```bash | ||
| cd examples | ||
| node authorize_and_store_papi.js | ||
| ``` | ||
|
|
||
| ## Key Differences from @polkadot/api | ||
|
|
||
| ### 1. Client Creation | ||
| **Old (@polkadot/api):** | ||
| ```javascript | ||
| const ws = new WsProvider('ws://localhost:10000'); | ||
| const api = await ApiPromise.create({ provider: ws }); | ||
| ``` | ||
|
|
||
| **New (PAPI):** | ||
| ```javascript | ||
| import { createClient } from 'polkadot-api'; | ||
| import { getWsProvider } from 'polkadot-api/ws-provider/node'; | ||
|
|
||
| const wsProvider = getWsProvider('ws://localhost:10000'); | ||
| const client = createClient(wsProvider); | ||
| const typedApi = client.getTypedApi(bulletin); | ||
| ``` | ||
|
|
||
| ### 2. Transactions | ||
| **Old:** | ||
| ```javascript | ||
| const tx = api.tx.transactionStorage.store(data); | ||
| const result = await tx.signAndSend(pair); | ||
| ``` | ||
|
|
||
| **New:** | ||
| ```javascript | ||
| const tx = typedApi.tx.TransactionStorage.store({ data }); | ||
| const result = await tx.signAndSubmit(pair); | ||
| ``` | ||
|
|
||
| ### 3. Type Safety | ||
| PAPI provides full TypeScript type safety based on your chain's metadata: | ||
| - Transaction parameters are type-checked | ||
| - Query results have proper types | ||
| - Auto-completion in IDEs | ||
|
|
||
| ### 4. Signing | ||
| PAPI uses a different signing interface. The `@polkadot-api/pjs-signer` package bridges between `@polkadot/keyring` and PAPI: | ||
|
|
||
| ```javascript | ||
| import { getPolkadotSignerFromPjs } from '@polkadot-api/pjs-signer'; | ||
| import { Keyring } from '@polkadot/keyring'; | ||
|
|
||
| const keyring = new Keyring({ type: 'sr25519' }); | ||
| const account = keyring.addFromUri('//Alice'); | ||
|
|
||
| // Create PAPI-compatible signer (simple!) | ||
| const signer = getPolkadotSignerFromPjs(account); | ||
| ``` | ||
|
|
||
| ## Benefits of PAPI | ||
|
|
||
| 1. **Type Safety**: Full TypeScript support with generated types | ||
| 2. **Light Client Support**: Can use smoldot for light client connections | ||
| 3. **Better Performance**: More efficient serialization/deserialization | ||
| 4. **Modern API**: Cleaner, more intuitive API design | ||
| 5. **Better Developer Experience**: Auto-completion and type checking | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Error: Cannot find module '@polkadot-api/descriptors' | ||
| Run: `npm run papi:generate` from the root of the repository | ||
|
|
||
| ### Connection issues | ||
| Make sure your bulletin chain node is running on ws://localhost:10000 | ||
|
|
||
| ### Metadata errors | ||
| If metadata changes, regenerate descriptors: `npm run papi:update` from the root of the repository | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| // npm install polkadot-api @polkadot-api/signer @polkadot/keyring @polkadot/util-crypto multiformats ipfs-http-client | ||
| // npx papi add -w ws://localhost:10000 bulletin | ||
| // ipfs daemon & | ||
| // ipfs swarm connect /ip4/127.0.0.1/tcp/10001/ws/p2p/12D3KooWQCkBm1BYtkHpocxCwMgR8yjitEeHGx8spzcDLGt2gkBm | ||
| // ipfs swarm connect /ip4/127.0.0.1/tcp/12347/ws/p2p/12D3KooWRkZhiRhsqmrQ28rt73K7V3aCBpqKugpsqfpggl4nzazpieyemw6xme | ||
| // ipfs swarm peers - should be there | ||
| // ipfs bitswap stat | ||
| // ipfs block get /ipfs/bafk2bzacebcnty2x5l3jr2sk5rvn7engdfkugpsqfpggl4nzazpieyemw6xme | ||
bkontur marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| import { createClient } from 'polkadot-api'; | ||
| import { Binary } from '@polkadot-api/substrate-bindings'; | ||
| import { getWsProvider } from 'polkadot-api/ws-provider/node'; | ||
bkontur marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| import { getPolkadotSigner } from '@polkadot-api/signer'; | ||
| import { Keyring } from '@polkadot/keyring'; | ||
| import { cryptoWaitReady } from '@polkadot/util-crypto'; | ||
| import { create } from 'ipfs-http-client'; | ||
| import { waitForNewBlock, cidFromBytes } from './common.js'; | ||
| import { bulletin } from '../.papi/descriptors/dist/index.mjs'; | ||
|
|
||
| async function authorizeAccount(typedApi, sudoPair, who, transactions, bytes) { | ||
| console.log('Creating authorizeAccount transaction...'); | ||
|
|
||
| const authorizeTx = typedApi.tx.TransactionStorage.authorize_account({ | ||
| who, | ||
| transactions, | ||
| bytes | ||
| }); | ||
|
|
||
| const sudoTx = typedApi.tx.Sudo.sudo({ | ||
| call: authorizeTx.decodedCall | ||
| }); | ||
|
|
||
| const result = await sudoTx.signAndSubmit(sudoPair); | ||
| console.log('Transaction authorizeAccount submitted:', result); | ||
| return result; | ||
| } | ||
|
|
||
| async function store(typedApi, pair, data) { | ||
| console.log('Storing data:', data); | ||
| const cid = cidFromBytes(data); | ||
|
|
||
| // Convert data to Uint8Array then wrap in Binary for PAPI typed API | ||
| const dataBytes = typeof data === 'string' ? | ||
| new Uint8Array(Buffer.from(data)) : | ||
| new Uint8Array(data); | ||
|
|
||
| // Wrap in Binary object for typed API - pass as object with 'data' property | ||
bkontur marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| const binaryData = Binary.fromBytes(dataBytes); | ||
| const tx = typedApi.tx.TransactionStorage.store({ data: binaryData }); | ||
|
|
||
| const result = await tx.signAndSubmit(pair); | ||
| console.log('Transaction store submitted:', result); | ||
|
|
||
| return cid; | ||
| } | ||
|
|
||
| // Connect to a local IPFS gateway (e.g. Kubo) | ||
| const ipfs = create({ | ||
| url: 'http://127.0.0.1:5001', // Local IPFS API | ||
| }); | ||
|
|
||
| async function read_from_ipfs(cid) { | ||
| // Fetch the block (downloads via Bitswap if not local) | ||
| console.log('Trying to get cid: ', cid); | ||
| try { | ||
| const block = await ipfs.block.get(cid, {timeout: 10000}); | ||
| console.log('Received block: ', block); | ||
| if (block.length !== 0) { | ||
| return block; | ||
| } | ||
| } catch (error) { | ||
| console.log('Block not found directly, trying cat...', error.message); | ||
| } | ||
|
|
||
| // Fetch the content from IPFS | ||
| console.log('Trying to chunk cid: ', cid); | ||
| const chunks = []; | ||
| for await (const chunk of ipfs.cat(cid)) { | ||
| chunks.push(chunk); | ||
| } | ||
|
|
||
| const content = Buffer.concat(chunks); | ||
| return content; | ||
| } | ||
|
|
||
| // Global client reference for cleanup | ||
| let client; | ||
|
|
||
| async function main() { | ||
| await cryptoWaitReady(); | ||
|
|
||
| // Create PAPI client with WebSocket provider | ||
| client = createClient(getWsProvider('ws://localhost:10000')); | ||
|
|
||
| // Get typed API with generated descriptors | ||
| const typedApi = client.getTypedApi(bulletin); | ||
|
|
||
| // Create keyring and accounts | ||
| const keyring = new Keyring({ type: 'sr25519' }); | ||
| const sudoAccount = keyring.addFromUri('//Alice'); | ||
| const whoAccount = keyring.addFromUri('//Alice'); | ||
|
|
||
| // Create PAPI-compatible signers using @polkadot-api/signer | ||
| // getPolkadotSigner expects (publicKey: Uint8Array, signingType, sign function) | ||
| const sudoSigner = getPolkadotSigner( | ||
| sudoAccount.publicKey, | ||
| 'Sr25519', | ||
| (input) => sudoAccount.sign(input) | ||
| ); | ||
| const whoSigner = getPolkadotSigner( | ||
| whoAccount.publicKey, | ||
| 'Sr25519', | ||
| (input) => whoAccount.sign(input) | ||
| ); | ||
|
|
||
| // Data | ||
| const who = whoAccount.address; | ||
| const transactions = 32; // u32 - regular number | ||
| const bytes = 64n * 1024n * 1024n; // u64 - BigInt for large numbers | ||
|
|
||
| console.log('Doing authorization...'); | ||
| await authorizeAccount(typedApi, sudoSigner, who, transactions, bytes); | ||
| await waitForNewBlock(); | ||
| console.log('Authorized!'); | ||
|
|
||
| console.log('Storing data ...'); | ||
| const dataToStore = "Hello, Bulletin with PAPI - " + new Date().toString(); | ||
| let cid = await store(typedApi, whoSigner, dataToStore); | ||
| console.log('Stored data with CID: ', cid); | ||
| await waitForNewBlock(); | ||
|
|
||
| console.log('Reading content... cid: ', cid); | ||
| let content = await read_from_ipfs(cid); | ||
| console.log('Content as bytes:', content); | ||
| console.log('Content as string:', content.toString()); | ||
|
|
||
| client.destroy(); | ||
| } | ||
|
|
||
| main().catch(console.error).finally(() => { | ||
| if (client) client.destroy(); | ||
| }); | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| { | ||
| "type": "module", | ||
| "scripts": { | ||
| "papi:generate": "papi add -w ws://localhost:10000 bulletin", | ||
| "papi:update": "papi" | ||
| }, | ||
| "dependencies": { | ||
| "@polkadot-api/descriptors": "file:.papi/descriptors", | ||
| "@polkadot-api/substrate-bindings": "^0.16.5", | ||
| "@polkadot/api": "^16.5.2", | ||
| "@polkadot/keyring": "^13.5.8", | ||
| "@polkadot/util": "^13.5.8", | ||
| "@polkadot/util-crypto": "^13.5.8", | ||
| "fs": "^0.0.1-security", | ||
| "ipfs-http-client": "^60.0.1", | ||
| "multiformats": "^13.4.1", | ||
| "polkadot-api": "^1.20.6", | ||
| "smoldot": "^2.0.39" | ||
| }, | ||
| "devDependencies": { | ||
| "@polkadot-api/cli": "^0.13.3" | ||
| } | ||
| } |
bkontur marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.