Skip to content

save proposal from archive node #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/gentle-masks-drum.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@agoric/synthetic-chain': patch
---

remove bankSend agd wrapper
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be a changeset for the save feature?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's part of the repo but not what gets a versioned release

4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -134,3 +134,7 @@ dist
Dockerfile
docker-bake.*
/upgrade-test-scripts

.aider*
!.aiderignore

4 changes: 3 additions & 1 deletion .prettierignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
*.json
proposals/87:fast-usdc-beta/submission/
submission
.aider*

7 changes: 5 additions & 2 deletions packages/synthetic-chain/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"node": "^18.19 || ^20.9"
},
"dependencies": {
"@agoric/client-utils": "0.1.1-dev-f9483e7.0",
"@endo/zip": "^1.0.9",
"better-sqlite3": "^11.10.0",
"chalk": "^5.4.1",
Expand All @@ -29,14 +30,16 @@
"tmp": "0.2.3"
},
"devDependencies": {
"@agoric/cosmic-proto": "0.5.0-u18.5",
"@agoric/cosmic-proto": "0.4.1-dev-c0e75ad.0",
Comment on lines -32 to +33
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why bump it backward?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's actually newer. the SDK release process bumped it, but u18 is older than dev which is every push to master.

fixing that is something Yarn 4 and Changesets will help with

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised the lerna dev publish doesn't bump versions the same way.

"@agoric/fast-usdc": "0.1.1-dev-f9483e7.0",
"@agoric/inter-protocol": "0.16.2-dev-f9483e7.0",
"@types/better-sqlite3": "^7.6.13",
"@types/glob": "^8.1.0",
"@types/node": "^22.15.17",
"@types/tmp": "0.2.6",
"ava": "^6.3.0",
"ts-blank-space": "^0.6.1",
"tsup": "^8.4.0",
"tsup": "^8.5.0",
"typescript": "^5.8.3"
},
"ava": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ set -e

PROPOSAL_PATH=$1

if [ -z "$PROPOSAL_PATH" ]; then
echo "Must specify what proposal to install"
exit 1
fi

# The base image is Node 16.9, which supports Corepack.
# Yarn v4 requires Node 18+ but so far it's working with 16.19.
export YARN_IGNORE_NODE=1
Expand All @@ -14,12 +19,10 @@ corepack enable
cd "$(dirname "$(realpath -- "$0")")"

# TODO consider yarn workspaces to install all in one command
if [ -n "$PROPOSAL_PATH" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my breaking change spidey-sense is tingling.

I presume this is internal, for now.

cd "../proposals/$PROPOSAL_PATH"

if test -f "yarn.lock"; then
yarn --version # only Berry supported, so next commands will fail on classic
yarn config set --home enableTelemetry 0
yarn install --immutable
fi
cd "../proposals/$PROPOSAL_PATH"

if test -f "yarn.lock"; then
yarn --version # only Berry supported, so next commands will fail on classic
yarn config set --home enableTelemetry 0
yarn install --immutable
fi
2 changes: 1 addition & 1 deletion packages/synthetic-chain/src/cli/build.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { execSync } from 'node:child_process';
import fs from 'node:fs';
import path from 'node:path';
import { ProposalInfo } from './proposals.js';
import { type ProposalInfo } from './proposals.js';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I wonder why yarn lint didn't catch this earlier.


export type Platform = 'linux/amd64' | 'linux/arm64';

Expand Down
142 changes: 142 additions & 0 deletions packages/synthetic-chain/src/cli/chain.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
import assert from 'node:assert';
import fsp from 'node:fs/promises';
import path from 'node:path';

import { makeTendermint34Client } from '@agoric/client-utils';
import { CoreEvalProposal } from '@agoric/cosmic-proto/agoric/swingset/swingset.js';
import { fromBase64 } from '@cosmjs/encoding';
import { decodeTxRaw } from '@cosmjs/proto-signing';
import { QueryClient, setupGovExtension } from '@cosmjs/stargate';
import { ProposalStatus } from 'cosmjs-types/cosmos/gov/v1beta1/gov.js';
import { copyFromCache } from '../lib/bundles.js';
import { isPassed, type ProposalInfo } from './proposals.js';

const DEFAULT_ARCHIVE_NODE = 'https://main-a.rpc.agoric.net:443';

// TODO use cosmic-proto to query, decoding into SearchTxsResultSDKType
type TxSearchResult = {
txs: Array<{
tx: string;
hash: string;
height: string;
}>;
total_count: string;
};

export async function fetchMsgInstallBundleTxs() {
const ACTION_TYPE = '/agoric.swingset.MsgInstallBundle';
// TODO: more configurable
let page = 1;
const perPage = 50;
const allMsgs: Array<{
height: string;
hash: string;
msg: any;
}> = [];

while (true) {
const query = `"message.action='${ACTION_TYPE}'"`;
const url = `${DEFAULT_ARCHIVE_NODE}/tx_search?query=${encodeURIComponent(query)}&page=${page}&per_page=${perPage}&order_by="desc"`;
const res = await fetch(url);

if (!res.ok) {
console.error(`Failed to fetch page ${page}: ${res.statusText}`);
break;
}

const data = await res.json();
const result: TxSearchResult = data.result;

if (result.total_count === '0' || !result.txs?.length) {
console.log('No more transactions found.');
break;
}

console.log(`Page ${page}: ${result.txs.length} transactions`);
for (const txEntry of result.txs) {
const decoded = decodeTxRaw(fromBase64(txEntry.tx));
for (const msg of decoded.body.messages) {
if (msg.typeUrl === ACTION_TYPE) {
allMsgs.push({
height: txEntry.height,
hash: txEntry.hash,
msg,
});
}
}
}

if (result.txs.length < perPage) break;
page++;
}

return allMsgs;
}

export async function saveProposalContents(proposal: ProposalInfo) {
assert(isPassed(proposal), 'unpassed propoosals are not on the chain');

const tm = await makeTendermint34Client(DEFAULT_ARCHIVE_NODE, { fetch });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: in a follow-on, I'm inclined to try factoring out this ambient authority using some tooling...

const queryClient = QueryClient.withExtensions(tm, setupGovExtension);

const { proposal: data } = await queryClient.gov.proposal(
proposal.proposalIdentifier,
);
console.log('Proposal data:', data);
assert.equal(data.proposalId, proposal.proposalIdentifier);
assert.equal(data.content?.typeUrl, proposal.type);
assert.equal(data.status, ProposalStatus.PROPOSAL_STATUS_PASSED);
switch (proposal.type) {
case '/agoric.swingset.CoreEvalProposal':
const something = CoreEvalProposal.fromProtoMsg(data.content as any);
console.log('Decoded proposal:', something);
const { evals } = something;
const submissionDir = path.join(
'proposals',
`${proposal.proposalIdentifier}:${proposal.proposalName}`,
'submission',
);
await fsp.mkdir(submissionDir, { recursive: true });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

likewise passing submissionDir as a string rather than an object representing filesystem access


// Save original core eval files
for (const [i, evalItem] of evals.entries()) {
const { jsonPermits, jsCode } = evalItem;
// Use index for unique filenames if proposalName is reused across multiple evals
const baseFilename = `${proposal.proposalName}${evals.length > 1 ? `-${i}` : ''}`;
await fsp.writeFile(
path.join(submissionDir, `${baseFilename}-permit.json`),
jsonPermits,
);
await fsp.writeFile(
path.join(submissionDir, `${baseFilename}.js`),
jsCode,
);
}
console.log('Proposal eval files saved to', submissionDir);

// Find and save referenced bundles
const allBundleIds = new Set<string>();
for (const { jsCode } of evals) {
const ids = jsCode.match(/b1-[a-z0-9]+/g);
if (ids) {
ids.forEach(id => allBundleIds.add(id));
}
}

if (allBundleIds.size > 0) {
console.log('Found referenced bundle IDs:', Array.from(allBundleIds));
for (const bundleId of allBundleIds) {
await copyFromCache(bundleId, submissionDir, console);
}
} else {
console.log('No bundle IDs found in proposal eval code.');
}
break;
case '/cosmos.params.v1beta1.ParameterChangeProposal':
console.log('Nothing to save for Parameter Change Proposal');
break;
case 'Software Upgrade Proposal':
console.warn('Nothing to save for Software Upgrade Proposal');
break;
}
}
2 changes: 1 addition & 1 deletion packages/synthetic-chain/src/cli/cli.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env node
#!/usr/bin/env -S node --import ts-blank-space/register
/**
* @file CLI entrypoint, transpiled during build so Node is the env to run it in
*/
Expand Down
2 changes: 1 addition & 1 deletion packages/synthetic-chain/src/cli/dockerfileGen.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import fs from 'node:fs';
import { Platform } from './build.js';
import { type Platform } from './build.js';
import {
encodeUpgradeInfo,
imageNameForProposal,
Expand Down
2 changes: 1 addition & 1 deletion packages/synthetic-chain/src/cli/doctor.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import fs from 'node:fs';
import path from 'node:path';
import { glob } from 'glob';
import { ProposalInfo } from './proposals.js';
import { type ProposalInfo } from './proposals.js';
import assert from 'node:assert';
import { execSync } from 'node:child_process';

Expand Down
2 changes: 1 addition & 1 deletion packages/synthetic-chain/src/cli/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { spawnSync, type SpawnSyncReturns } from 'node:child_process';
import { existsSync, realpathSync } from 'node:fs';
import { resolve as resolvePath } from 'node:path';
import { fileSync as createTempFile } from 'tmp';
import { ProposalInfo, imageNameForProposal } from './proposals.js';
import { type ProposalInfo, imageNameForProposal } from './proposals.js';

const createMessageFile = (proposal: ProposalInfo) =>
createTempFile({ prefix: proposal.proposalName });
Expand Down
12 changes: 1 addition & 11 deletions packages/synthetic-chain/src/lib/agd-lib.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
import assert from 'node:assert';
import type { ExecFileSyncOptionsWithStringEncoding } from 'node:child_process';
import { CHAINID, VALIDATORADDR } from './constants.js';
import { agd } from './cliHelper.js';

const { freeze } = Object;

Expand Down Expand Up @@ -54,6 +52,7 @@ export const makeAgd = ({
query: async (
qArgs:
| [kind: 'gov', domain: string, ...rest: any]
| [kind: 'txs', filter: string]
| [kind: 'tx', txhash: string]
| [mod: 'vstorage', kind: 'data' | 'children', path: string],
) => {
Expand Down Expand Up @@ -142,12 +141,3 @@ export const makeAgd = ({
};
return make();
};

export const bankSend = (addr: string, wanted: string) => {
const chain = ['--chain-id', CHAINID];
const from = ['--from', VALIDATORADDR];
const testKeyring = ['--keyring-backend', 'test'];
const noise = [...from, ...chain, ...testKeyring, '--yes'];

return agd.tx('bank', 'send', VALIDATORADDR, addr, wanted, ...noise);
};
73 changes: 73 additions & 0 deletions packages/synthetic-chain/src/lib/bundles.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import assert from 'node:assert/strict';
import { copyFile, writeFile } from 'node:fs/promises';
import { gunzipSync } from 'node:zlib';

import { agoric } from '@agoric/cosmic-proto/agoric/bundle.js';
import type { MsgInstallBundle } from '@agoric/cosmic-proto/agoric/swingset/msgs.js';
import path from 'node:path';

const CACHE_DIR = path.join(
process.env.HOME || process.env.USERPROFILE || '',
'.agoric',
'cache',
);
Comment on lines +9 to +13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this CACHE_DIR calculation up to scripts/fetch-all-bundles.ts would make the data flow more clear (and avoid ambient authority).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it but then it made the cache dir configurable in a way that I didn't want. I also realized it was conflating authority with configuration. What would actually be ocap would be removing fs and path from the bundles module, and creating an ad-hoc powers API for it to use, but I don't think overall that's warranted


export async function base64ToBlob(
base64: string,
type = 'application/octet-stream',
): Promise<Blob> {
return fetch(`data:${type};base64,${base64}`).then(res => res.blob());
}

export async function decompressBlob(blob: Blob): Promise<Blob> {
const ds = new DecompressionStream('gzip');
const decompressedStream = blob.stream().pipeThrough(ds);
return new Response(decompressedStream).blob();
}

export async function bundleInMessage(msg: MsgInstallBundle) {
const { compressedBundle: b64gzip, uncompressedSize: size } = msg;
const bundleText = Buffer.from(gunzipSync(b64gzip)).toString('utf8');
assert.equal(bundleText.length, Number(size), 'bundle size mismatch');
return { bundleText, size };
}

export async function writeInstalledBundle(
basePath: string,
msgInstall: MsgInstallBundle,
{ log } = { log: (...msgs) => {} },
) {
const bundle = await bundleInMessage(msgInstall);
const bundleObj = JSON.parse(bundle.bundleText);
const filename = path.join(
basePath,
`b1-${bundleObj.endoZipBase64Sha512}.json`,
);
await writeFile(filename, bundle.bundleText, 'utf8');
log(`Wrote bundleText to ${filename}`);
}

export async function refreshBundlesCache(
txs: Array<{
hash: string;
height: string;
msg: any;
}>,
) {
for (const tx of txs) {
console.log(`\nBlock: ${tx.height}, TxHash: ${tx.hash}`);
const msgInstall = agoric.swingset.MsgInstallBundle.fromProtoMsg(tx.msg);
await writeInstalledBundle(CACHE_DIR, msgInstall, console);
}
}

export async function copyFromCache(
bundleId: string,
targetDir: string,
{ log } = { log: (...msgs) => {} },
) {
const bundlePath = path.join(CACHE_DIR, `${bundleId}.json`);
const targetPath = path.join(targetDir, `${bundleId}.json`);
await copyFile(bundlePath, targetPath);
log(`Copied bundle from ${bundlePath} to ${targetPath}`);
}
7 changes: 5 additions & 2 deletions packages/synthetic-chain/src/lib/cliHelper.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import { $, execaCommand } from 'execa';
import { BINARY, SDK_ROOT } from './constants.js';
import type { Options as ExecaOptions } from 'execa';
import { $, execaCommand } from 'execa';
import { NonNullish } from './assert.js';
import { SDK_ROOT } from './constants.js';

export const BINARY = NonNullish(process.env.binary);

export const executeCommand = async (
command: string,
Expand Down
6 changes: 3 additions & 3 deletions packages/synthetic-chain/src/lib/commonUpgradeHelpers.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { $, type TemplateExpression } from 'execa';
import assert from 'node:assert';
import fsp from 'node:fs/promises';
import * as path from 'node:path';
import { agd, agoric, agops } from './cliHelper.js';
import { CHAINID, HOME, VALIDATORADDR } from './constants.js';
import assert from 'node:assert';
import { agd, agops, agoric } from './cliHelper.js';
import { CHAINID, VALIDATORADDR } from './constants.js';

const waitForBootstrap = async (): Promise<number> => {
const endpoint = 'localhost';
Expand Down
Loading