use embedded dmsghttp config for dmsg utilities [WIP] #298

0pcom · 2025-03-01T15:46:30Z

this attempts to add to dmsg functionality that currently exists for skywire. Using the embedded dmsg servers of the dmsghttp-config.json it should be possible to connect to dmsg without http request to the dmsg discovery.

Testing

$ go run cmd/dmsgcurl/dmsgcurl.go -l debug dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health

the default behavior is to avoid http requests. The -z flag switches to using http request & the discovery.

It's not yet working

$ go run cmd/dmsgcurl/dmsgcurl.go -l debug dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health
[2025-03-01T09:46:54.452021841-06:00] DEBUG direct.NewClient [dmsgcurl]: Created Direct client.
[2025-03-01T09:46:54.452436651-06:00] DEBUG [dmsgcurl]: Connecting to dmsg network... public_key="0397d037b74de2a07d14dadc1be152f50a11906f3536ed7d863c68fac87d64e7eb"
[2025-03-01T09:46:54.45248061-06:00] DEBUG [dmsgcurl]: Discovering dmsg servers...
[2025-03-01T09:46:54.452528976-06:00] DEBUG [dmsgcurl]: Dialing session... remote_pk=0281a102c82820e811368c8d028cf11b1a985043b726b1bcdb8fce89b27384b2cb
[2025-03-01T09:46:54.724526124-06:00] DEBUG [dmsgcurl]: Updating entry. entry=	version: 
	sequence: 0
	registered at: 0
	static public key: 0397d037b74de2a07d14dadc1be152f50a11906f3536ed7d863c68fac87d64e7eb
	signature: 
	entry is registered as client. Related info: 
		delegated servers: 
			0281a102c82820e811368c8d028cf11b1a985043b726b1bcdb8fce89b27384b2cb


[2025-03-01T09:46:54.724641402-06:00] DEBUG [dmsgcurl]: Serving session. remote_pk=0281a102c82820e811368c8d028cf11b1a985043b726b1bcdb8fce89b27384b2cb
[2025-03-01T09:46:54.724724866-06:00] DEBUG [dmsgcurl]: Dmsg network ready.
[2025-03-01T09:46:54.724886386-06:00] ERROR [dmsgcurl]: error="failed to connect to HTTP server: Get "dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health": dmsg error 102 - entry is not of client in discovery"

0pcom · 2025-03-03T16:58:37Z

@mrpalide I have significantly refactored the code for dmsgcurl. Can you check on this error? I can't seem to figure it out

This will produce the error

go run cmd/dmsgcurl/dmsgcurl.go -l debug dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health
[2025-03-03T10:56:22.935355486-06:00] DEBUG direct.NewClient [dmsgcurl]: Created Direct client.
[2025-03-03T10:56:22.935438229-06:00] DEBUG [dmsgcurl]: Connecting to dmsg network...
 public_key="035fae5800a17444945de9e4abb253d092746e6227af576ae1b65246442042bf2b"
[2025-03-03T10:56:22.935473263-06:00] DEBUG [dmsgcurl]: Discovering dmsg servers...
[2025-03-03T10:56:22.935502653-06:00] DEBUG [dmsgcurl]: Dialing session... remote_pk=0371ab4bcff7b121f4b91f6856d6740c6f9dc1fe716977850aeb5d84378b300a13
[2025-03-03T10:56:23.202575287-06:00] DEBUG [dmsgcurl]: Updating entry.
 entry=	version: 
	sequence: 0
	registered at: 0
	static public key: 035fae5800a17444945de9e4abb253d092746e6227af576ae1b65246442042bf2b
	signature: 
	entry is registered as client. Related info: 
		delegated servers: 
			0371ab4bcff7b121f4b91f6856d6740c6f9dc1fe716977850aeb5d84378b300a13


[2025-03-03T10:56:23.202782596-06:00] DEBUG [dmsgcurl]: Serving session. remote_pk=0371ab4bcff7b121f4b91f6856d6740c6f9dc1fe716977850aeb5d84378b300a13
[2025-03-03T10:56:23.202810377-06:00] DEBUG [dmsgcurl]: Dmsg network ready.
[2025-03-03T10:56:23.202942004-06:00] ERROR [dmsgcurl]: Failed to preform HTTP request
 error="Get "dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health": dmsg error 102 - entry is not of client in discovery"
[2025-03-03T10:56:23.203092327-06:00] DEBUG [dmsgcurl]: Stopped serving client!
[2025-03-03T10:56:23.203106074-06:00] DEBUG [dmsgcurl]: Stopped accepting streams. error="session shutdown" session=0371ab4bcff7b121f4b91f6856d6740c6f9dc1fe716977850aeb5d84378b300a13
[2025-03-03T10:56:23.203277854-06:00] DEBUG [dmsgcurl]: Session closed. error=<nil>
[2025-03-03T10:56:23.203351526-06:00] DEBUG [dmsgcurl]: All sessions closed.
[2025-03-03T10:56:23.203370311-06:00] DEBUG [dmsgcurl]: Deleting entry.
 entry=	version: 
	sequence: 0
	registered at: 0
	static public key: 035fae5800a17444945de9e4abb253d092746e6227af576ae1b65246442042bf2b
	signature: 
	entry is registered as client. Related info: 
		delegated servers: 
			0371ab4bcff7b121f4b91f6856d6740c6f9dc1fe716977850aeb5d84378b300a13


[2025-03-03T10:56:23.203499434-06:00] DEBUG [dmsgcurl]: Entry Deleted successfully.
[2025-03-03T10:56:23.203515467-06:00] DEBUG [dmsgcurl]: Disconnected from dmsg network.
 error=<nil>
Failed to write to log, write /dev/stdout: file already closed
Failed to write to log, write /dev/stdout: file already closed
2025/03/03 10:56:23 Failed to execute command: failed to preform HTTP request: Get "dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health": dmsg error 102 - entry is not of client in discovery
exit status 1

Error not present with the -z flag which is the original / previous behavior of dmsgcurl, using the dmsg-discovery as normal

go run cmd/dmsgcurl/dmsgcurl.go -zl debug dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health
[2025-03-03T10:57:35.441712138-06:00] DEBUG [dmsgcurl]: DMSG Discovery: [http://dmsgd.skywire.skycoin.com]
[2025-03-03T10:57:35.442084971-06:00] DEBUG disc.NewHTTP [dmsgcurl]: Created HTTP client. addr="http://dmsgd.skywire.skycoin.com"
[2025-03-03T10:57:35.44211256-06:00] DEBUG [dmsgcurl]: Connecting to dmsg network...
 dmsg_disc="http://dmsgd.skywire.skycoin.com"
[2025-03-03T10:57:35.442127911-06:00] DEBUG [dmsgcurl]: 
 public_key="03c1c9df6afd866efd5b52cbbd36efb3bd50c113aa4b11282223cc58182a6ebe65"
[2025-03-03T10:57:35.442153399-06:00] DEBUG [dmsg_client]: Discovering dmsg servers...
[2025-03-03T10:57:35.709116117-06:00] DEBUG [dmsg_client]: Dialing session... remote_pk=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2025-03-03T10:57:35.991984933-06:00] DEBUG [dmsgcurl]: Dmsg network ready.
[2025-03-03T10:57:35.992003029-06:00] DEBUG [dmsg_client]: Serving session. remote_pk=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
{"dmsg_address":"036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80","dmsg_discovery":"https://dmsgd.skywire.skycoin.com","frontend_run_time":"45h26m24.945704867s","frontend_start_time":"2025-03-01T13:31:11.623621146-06:00","reward_system_active":"inactive","reward_system_next_run":"Mon 2025-03-03 11:30:06 CST; 32min left","reward_system_prev_run_duration":"   Duration: 9min 11.981s","whitelisted_keys":["03b32e6e4bee60441aab3e09db69bc2f0b9ec60b4bff233283211bbc2e0ac58114","0254a898d43b840f6320028be1f9266d64e186bb48f462dafed87c601ea67911e7","03937892ae1a60459ad79dcf181ab6741c68c4c077878a9afabd5b91757115f87d"]}[2025-03-03T10:57:36.584598747-06:00] DEBUG [dmsg_client]: Stopped serving client!
[2025-03-03T10:57:36.584646948-06:00] DEBUG [dmsg_client]: Session closed. error=<nil>
[2025-03-03T10:57:36.584683551-06:00] DEBUG [dmsg_client]: Stopped accepting streams. error="session shutdown" session=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2025-03-03T10:57:36.584684648-06:00] DEBUG [dmsg_client]: All sessions closed.
[2025-03-03T10:57:36.723147653-06:00] DEBUG [dmsg_client]: Deleting entry.
 entry=	version: 0.0.1
	sequence: 0
	registered at: 1741021055857963369
	static public key: 03c1c9df6afd866efd5b52cbbd36efb3bd50c113aa4b11282223cc58182a6ebe65
	signature: 6468478286c5c0ea07904f8bf57c649d7d07ee7c0bef120a2536f099082e3a0e718aab67632dc522fcf081a02ac4f6a8ae7913fb9e640c852e3b3e325269c67d00
	entry is registered as client. Related info: 
		delegated servers: 
			0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7


[2025-03-03T10:57:36.857233925-06:00] DEBUG [dmsg_client]: Entry Deleted successfully.
[2025-03-03T10:57:36.857280297-06:00] DEBUG [dmsgcurl]: Disconnected from dmsg network.
 error=<nil>

mrpalide · 2025-05-04T14:04:51Z

Seems solved.

ubuntu@ubuntu-c1-medium1-goethe-1:~/dmsg$ go run cmd/dmsgcurl/dmsgcurl.go -l debug dmsg://036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80/health
[2025-05-04T14:04:03.350712693Z] DEBUG direct.NewClient [dmsgcurl]: Created Direct client.
[2025-05-04T14:04:03.351030216Z] DEBUG [dmsgcurl]: Connecting to dmsg network...
 public_key="02243562411d4a0655178cb4c8af2386ce31ca982347bf1da37d0b87d563c7a8ee"
[2025-05-04T14:04:03.351268428Z] DEBUG [dmsgcurl]: Discovering dmsg servers...
[2025-05-04T14:04:03.35157431Z] DEBUG [dmsgcurl]: Dialing session... remote_pk=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2025-05-04T14:04:03.701429072Z] DEBUG [dmsgcurl]: Updating entry.
 entry=	version:
	sequence: 0
	registered at: 0
	static public key: 02243562411d4a0655178cb4c8af2386ce31ca982347bf1da37d0b87d563c7a8ee
	signature:
	entry is registered as client. Related info:
		delegated servers:
			0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7


[2025-05-04T14:04:03.70388273Z] DEBUG [dmsgcurl]: Serving session. remote_pk=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2025-05-04T14:04:03.703919063Z] DEBUG [dmsgcurl]: Dmsg network ready.
{"dmsg_address":"036a70e6956061778e1883e928c1236189db14dfd446df23d83e45c321b330c91f:80","dmsg_discovery":"https://dmsgd.skywire.skycoin.com","frontend_run_time":"41h9m39.742677838s","frontend_start_time":"2025-05-02T15:54:24.634384004-05:00","reward_system_active":"inactive","reward_system_next_run":"Sun 2025-05-04 09:42:35 CDT; 38min left","reward_system_prev_run_duration":"   Duration: 9min 34.501s","whitelisted_keys":["03b32e6e4bee60441aab3e09db69bc2f0b9ec60b4bff233283211bbc2e0ac58114","0254a898d43b840f6320028be1f9266d64e186bb48f462dafed87c601ea67911e7","03937892ae1a60459ad79dcf181ab6741c68c4c077878a9afabd5b91757115f87d"]}[2025-05-04T14:04:04.500047499Z] DEBUG [dmsgcurl]: Stopped serving client!
[2025-05-04T14:04:04.500318794Z] DEBUG [dmsgcurl]: Stopped accepting streams. error="session shutdown" session=0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7
[2025-05-04T14:04:04.501048961Z] DEBUG [dmsgcurl]: Session closed. error=<nil>
[2025-05-04T14:04:04.501413212Z] DEBUG [dmsgcurl]: All sessions closed.
[2025-05-04T14:04:04.501803507Z] DEBUG [dmsgcurl]: Deleting entry.
 entry=	version:
	sequence: 0
	registered at: 0
	static public key: 02243562411d4a0655178cb4c8af2386ce31ca982347bf1da37d0b87d563c7a8ee
	signature:
	entry is registered as client. Related info:
		delegated servers:
			0326978f5a53aff537dbb47fed58b1f123af3b00132d365f1309a14db4168dcff7


[2025-05-04T14:04:04.504668937Z] DEBUG [dmsgcurl]: Entry Deleted successfully.
[2025-05-04T14:04:04.504832091Z] DEBUG [dmsgcurl]: Disconnected from dmsg network.
 error=<nil>
ubuntu@ubuntu-c1-medium1-goethe-1:~/dmsg$

0pcom · 2025-05-06T22:40:01Z

@mrpalide can you test this when you get a chance. The server side doesn't seem to work via dmsg-direct and I can't figure out why.

skywire cli config gen-keys > dmsghttp.key
go run cmd/dmsg/dmsg.go http -e7 -s $(tail -n1 dmsghttp.key)

Then in a different terminal try to fetch the README.md from the server public key

go run cmd/dmsg/dmsg.go curl -l debug -e7 dmsg://$(head -n1 dmsghttp.key):80/README.md

That fails, for me, but if I switch the server to use the discovery it does work

go run cmd/dmsg/dmsg.go http -e7 -z -s $(tail -n1 dmsghttp.key)

0pcom added 10 commits March 1, 2025 09:43

implement dmsg-direct using embedded dmsghttp config for dmsgcurl

405a96c

continue dmsghttp-config implementation for dmsgcurl

b83d5c8

slight refactorization

f93ab62

fix typo

1344e26

improve logging

9760f13

continue improvements to logging

3eb3c01

continue improvements to logging

c7beac0

continue improvements to logging

99e8331

remove unneeded logging

6485060

fix typo

e50c9bb

0pcom and others added 7 commits March 3, 2025 11:00

fix CI errors

ad08960

hotfix merge conflicts

041ad45

update mohammed's revised error implementation

68cec0c

fix spelling

b4c6743

merge develop ; fix merge conflicts

ded7efa

improve a part of code, as possible solution

5ff2f0e

remove ineffectual assignement

2b81d28

mrpalide and others added 5 commits May 4, 2025 14:06

check error on PostEntry of destination

b5aebac

remove //TODO comment

b3f1d5d

standardize var naming, begin dmsghttp, add proxy for direct

50ee95f

continue implementing dmsg-direct for dmsghttp filee server

4c8b696

change destination pk

c83baa0

mrpalide and others added 5 commits May 7, 2025 12:41

fix dmsgSession issue on curl

065c036

fix CI issues

fa2b66c

implement dmsg-direct for dmsgip command

ca7338d

update dmsgsocks & dmsgweb to use dmsg-direect by default

0719222

make format

570d807

0pcom added 4 commits May 7, 2025 09:41

add error checks

74c37c1

check for errors when conneceting to dmsg

a7e50b5

fix return

5485dea

remove unused

f0aa468

0pcom merged commit f0a9a2e into skycoin:develop May 7, 2025
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

use embedded dmsghttp config for dmsg utilities [WIP] #298

use embedded dmsghttp config for dmsg utilities [WIP] #298

Uh oh!

0pcom commented Mar 1, 2025 •

edited

Loading

Uh oh!

0pcom commented Mar 3, 2025

Uh oh!

mrpalide commented May 4, 2025

Uh oh!

0pcom commented May 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

use embedded dmsghttp config for dmsg utilities [WIP] #298

use embedded dmsghttp config for dmsg utilities [WIP] #298

Uh oh!

Conversation

0pcom commented Mar 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Uh oh!

0pcom commented Mar 3, 2025

Uh oh!

mrpalide commented May 4, 2025

Uh oh!

0pcom commented May 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

0pcom commented Mar 1, 2025 •

edited

Loading