Add examples

[koalalorenzo]

This project looks promising :)
It would be nice to add some examples to how to use it

[hsanjuan]

I'm not sure I wanted to commit this but since I did, it might be useful until I write a better example: https://github.com/hsanjuan/ipfs-lite/blob/d2191f217f81879baf5e5d09d51d5233628ac430/test/ipfs-lite.go

[bonedaddy]

@hsanjuan The example is giving me the following output

02:12:02.368 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
02:12:02.368 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
02:12:02.369 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
02:12:02.369 ERROR   ipfslite: dial attempt failed: no good addresses ipfs.go:122
Hello World

What's the error message being caused by?

[hsanjuan]

hey @postables that's probably because some of the default bootstrap peers are offline or don't work when it tries to connect to them. Maybe worth reducing this to a warning... however it does connect correctly to other bootstrap peers and as you see it was able to fetch the content from somewhere.

[bonedaddy]

ah okay that makes sense, thanks!

[probonopd]

Getting

me@host:~$ go run /home/me/go/src/github.com/hsanjuan/ipfs-lite/examples/litepeer/litepeer.go 
2019-10-27T20:03:00.795+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2a03:b0c0:0:1010::23:1001/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
  * [/ip4/178.62.158.247/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
2019-10-27T20:03:00.943+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:a880:800:10::4a:5001/tcp/4001] dial tcp6 [2604:a880:800:10::4a:5001]:4001: connect: connection refused
  * [/ip4/104.236.76.40/tcp/4001] dial tcp4 104.236.76.40:4001: connect: connection refused
2019-10-27T20:03:01.074+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:a880:1:20::203:d001/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
  * [/ip4/104.236.179.241/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
2019-10-27T20:03:01.079+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip4/104.131.131.82/tcp/4001] dial tcp4 104.131.131.82:4001: connect: connection refused
2019-10-27T20:03:01.121+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip4/128.199.219.111/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
  * [/ip6/2400:6180:0:d0::151:6001/tcp/4001] failed to negotiate security protocol: rsa keys must be >= 2048 bits to be useful
2019-10-27T20:03:05.798+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:2000:7a00::1/tcp/4001] dial tcp6 [2604:1380:2000:7a00::1]:4001: connect: connection refused
  * [/ip4/147.75.83.83/tcp/4001] dial tcp4 147.75.83.83:4001: connect: connection refused
  * [/ip4/139.178.70.249/tcp/4001] dial tcp4 139.178.70.249:4001: connect: connection refused
  * [/ip6/2604:1380:1000:6000::7/tcp/4001] dial tcp6 [2604:1380:1000:6000::7]:4001: i/o timeout
2019-10-27T20:03:05.798+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:1000:6000::1/tcp/4001] dial tcp6 [2604:1380:1000:6000::1]:4001: connect: connection refused
  * [/ip4/139.178.68.71/tcp/4001] dial tcp4 139.178.68.71:4001: connect: connection refused
  * [/ip6/2604:1380:1000:6000::3/tcp/4001] dial tcp6 [2604:1380:1000:6000::3]:4001: i/o timeout
2019-10-27T20:03:05.799+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:0:c100::1/tcp/4001] dial tcp6 [2604:1380:0:c100::1]:4001: connect: connection refused
  * [/ip4/147.75.77.187/tcp/4001] dial tcp4 147.75.77.187:4001: connect: connection refused
  * [/ip4/139.178.70.181/tcp/4001] dial tcp4 139.178.70.181:4001: connect: connection refused
  * [/ip6/2604:1380:1000:6000::5/tcp/4001] dial tcp6 [2604:1380:1000:6000::5]:4001: i/o timeout
2019-10-27T20:03:05.800+0100	WARN	ipfslite	go-log/log.go:187	failed to dial : all dials failed
  * [/ip6/2604:1380:3000:1f00::1/tcp/4001] dial tcp6 [2604:1380:3000:1f00::1]:4001: connect: connection refused
  * [/ip4/147.75.94.115/tcp/4001] dial tcp4 147.75.94.115:4001: connect: connection refused
  * [/ip4/139.178.88.21/tcp/4001] dial tcp4 0.0.0.0:4005->139.178.88.21:4001: i/o timeout
  * [/ip6/2604:1380:1000:6000::9/tcp/4001] dial tcp6 [2604:1380:1000:6000::9]:4001: i/o timeout
2019-10-27T20:03:05.800+0100	WARN	ipfslite	go-log/log.go:192	only connected to 0 bootstrap peers out of 9
2019-10-27T20:03:05.800+0100	WARN	dht	go-log/log.go:192	self walk: error: failed to find any peer in table

What is going wrong here? Can the code be changed so that the default bootstrap peer list from https://docs.ipfs.io/guides/examples/bootstrap/ is used?

[hsanjuan]

Hi, the problem is libp2p was upgraded to only allow 2048 rsa keys and the default bootstrappers use 1024 ones.

You can either:

• set crypto.MinRsaKeyBits (https://github.com/libp2p/go-libp2p-core/blob/master/crypto/rsa_common.go#L13) to 1024
• Use any other bootstrappers
• Wait, the team is working to resolve this by launching more bootstrappers that were already in the list:
  The IPFS bootstrap nodes seem to use 1024 bits RSA keys ipfs/infra#378

I'm going to update the example with the first fix though...

[probonopd]

Maybe it would be a good idea to extend the example to also write (add) something to IPFS.
But wait, why are we getting a different CID for the "Hello World"?

package main

// This example launches an IPFS-Lite peer, adds a hello-world to
// and fetches a hello-world hash from the IPFS network.

import (
	"context"
	"fmt"
	"io/ioutil"
	"strings"

	ipfslite "github.com/hsanjuan/ipfs-lite"
	"github.com/ipfs/go-cid"
	corecrypto "github.com/libp2p/go-libp2p-core/crypto"
	crypto "github.com/libp2p/go-libp2p-crypto"
	"github.com/multiformats/go-multiaddr"
)

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Bootstrappers are using 1024 keys. See:
	// https://github.com/ipfs/infra/issues/378
	corecrypto.MinRsaKeyBits = 1024

	ds, err := ipfslite.BadgerDatastore("test")
	if err != nil {
		panic(err)
	}
	priv, _, err := crypto.GenerateKeyPair(crypto.RSA, 2048)
	if err != nil {
		panic(err)
	}

	listen, _ := multiaddr.NewMultiaddr("/ip4/0.0.0.0/tcp/4005")

	h, dht, err := ipfslite.SetupLibp2p(
		ctx,
		priv,
		nil,
		[]multiaddr.Multiaddr{listen},
	)

	if err != nil {
		panic(err)
	}

	lite, err := ipfslite.New(ctx, ds, h, dht, nil)
	if err != nil {
		panic(err)
	}

	lite.Bootstrap(ipfslite.DefaultBootstrapPeers())

	fmt.Println("Adding file")
	n, err := lite.AddFile(ctx, strings.NewReader("Hello World"), nil)
	if err != nil {
		panic(err)
	}

	fmt.Println(n.Cid())
	// Why don't we get "QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u"?

	fmt.Println("")
	cc := "QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u"
	fmt.Println("Getting", cc)

	c, _ := cid.Decode(cc)
	rsc, err := lite.GetFile(ctx, c)
	if err != nil {
		panic(err)
	}
	defer rsc.Close()
	content, err := ioutil.ReadAll(rsc)
	if err != nil {
		panic(err)
	}

	fmt.Println(string(content))
}

[hsanjuan]

Line-breaks?

…

-------- Original Message --------

On Oct 28, 2019, 21:13, probonopd wrote: Maybe it would be a good idea to extend the example to also write (add) something to IPFS. But wait, why are we getting a different CID for the "Hello World"? package main // This example launches an IPFS-Lite peer, adds a hello-world to // and fetches a hello-world hash from the IPFS network. import ( "context" "fmt" "io/ioutil" "strings" ipfslite "github.com/hsanjuan/ipfs-lite" "github.com/ipfs/go-cid" corecrypto "github.com/libp2p/go-libp2p-core/crypto" crypto "github.com/libp2p/go-libp2p-crypto" "github.com/multiformats/go-multiaddr" ) func main() { ctx, cancel := context.WithCancel(context.Background()) defer cancel() // Bootstrappers are using 1024 keys. See: // ipfs/infra#378 corecrypto.MinRsaKeyBits = 1024 ds, err := ipfslite.BadgerDatastore("test") if err != nil { panic(err) } priv, _, err := crypto.GenerateKeyPair(crypto.RSA, 2048) if err != nil { panic(err) } listen, _ := multiaddr.NewMultiaddr("/ip4/0.0.0.0/tcp/4005") h, dht, err := ipfslite.SetupLibp2p( ctx, priv, nil, []multiaddr.Multiaddr{listen}, ) if err != nil { panic(err) } lite, err := ipfslite.New(ctx, ds, h, dht, nil) if err != nil { panic(err) } lite.Bootstrap(ipfslite.DefaultBootstrapPeers()) fmt.Println("Adding file") n, err := lite.AddFile(ctx, strings.NewReader("Hello World"), nil) if err != nil { panic(err) } fmt.Println(n.Cid()) // Why don't we get "QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u"? fmt.Println("") cc := "QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u" fmt.Println("Getting", cc) c, _ := cid.Decode(cc) rsc, err := lite.GetFile(ctx, c) if err != nil { panic(err) } defer rsc.Close() content, err := ioutil.ReadAll(rsc) if err != nil { panic(err) } fmt.Println(string(content)) } — You are receiving this because you were mentioned. Reply to this email directly, [view it on GitHub](#1?email_source=notifications&email_token=AAH2XTQ3ESJEMRX5GKIEXVDQQ5BWFA5CNFSM4HAOEN2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECOHQ2I#issuecomment-547125353), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AAH2XTTT5C7JVVTL4YOPHE3QQ5BWFANCNFSM4HAOEN2A).

[probonopd]

To answer my own question, we don't get a "Qm..." hash because there is apparently a new has format now that does not begin with "Qm...".

Follow-up question:
How can the example be made to actually serve the file? Right now the example just exits, and peers cannot download the added file.

[hsanjuan]

@probonopd ah, yes I guess it uses CIDv1 by default, instead of v0.

How can the example be made to actually serve the file? Right now the example just exits, and peers cannot download the added file.

You need to keep the peer (the example) running then.. I think adding select{ } at the end might make it.

[probonopd]

Can you please post a working example? This does not seem to work for me:

package main

// This example launches an IPFS-Lite peer, adds a hello-world to
// and fetches a hello-world hash from the IPFS network.

import (
	"context"
	"fmt"
	"io/ioutil"
	"os"

	ipfslite "github.com/hsanjuan/ipfs-lite"
	"github.com/ipfs/go-cid"
	corecrypto "github.com/libp2p/go-libp2p-core/crypto"
	crypto "github.com/libp2p/go-libp2p-crypto"
	"github.com/multiformats/go-multiaddr"
)

func main() {
	ctx, cancel := context.WithCancel(context.Background())
	defer cancel()

	// Bootstrappers are using 1024 keys. See:
	// https://github.com/ipfs/infra/issues/378
	corecrypto.MinRsaKeyBits = 1024

	fmt.Println("Generating datastore")

	ds, err := ipfslite.BadgerDatastore("test")
	if err != nil {
		panic(err)
	}
	priv, _, err := crypto.GenerateKeyPair(crypto.RSA, 2048)
	if err != nil {
		panic(err)
	}

	fmt.Println("Listening")

	listen, _ := multiaddr.NewMultiaddr("/ip4/0.0.0.0/tcp/4005")

	h, dht, err := ipfslite.SetupLibp2p(
		ctx,
		priv,
		nil,
		[]multiaddr.Multiaddr{listen},
		ipfslite.Libp2pOptionsExtra...,
	)

	if err != nil {
		panic(err)
	}

	lite, err := ipfslite.New(ctx, ds, h, dht, nil)
	if err != nil {
		panic(err)
	}

        fmt.Println("Peer address", h.ID())
	fmt.Println("Bootstrapping")

	lite.Bootstrap(ipfslite.DefaultBootstrapPeers())

	fmt.Println("Adding file")

	f, err := os.Open("/etc/os-release")
	if err != nil {
		fmt.Println(err)
	} else {

		n, err := lite.AddFile(ctx, f, nil)
		if err != nil {
			fmt.Println(err)
		}

		f.Close()

		fmt.Println("Added, check it at https://explore.ipld.io/#/explore/" + n.String())

	}
	fmt.Println("")
	cc := "bafybeiduiecxoeiqs3gyc6r7v3lymmhserldnpw62qjnhmqsulqjxjmtzi"
	fmt.Println("Getting", cc)

	c, _ := cid.Decode(cc)
	rsc, err := lite.GetFile(ctx, c)
	if err != nil {
		fmt.Println(err)
	}
	defer rsc.Close()
	content, err := ioutil.ReadAll(rsc)
	if err != nil {
		fmt.Println(err)
	}

	fmt.Println(string(content))

	// Run forever. Is this sufficient to serve the files?

	ch := make(chan struct{})
	<-ch
}

[hsanjuan]

@probonopd that seems enough, but mind that your nodes needs to be online for a while to become well connected and should be reachable by others etc.

[probonopd]

Unfortunately it did not seem to work for me. I also cannot access the file from the browser on localhost.

[hsanjuan]

what if you manually connect your local ipfs daemon to /ip4/127.0.0.1/tcp/4005/ipfs/<the generated peer id-you will have to print h.ID() to get it> ?

The example does not run the mDNS discovery service so it can only be reached from dht and that may not work very well for an spontaneous peer (and the ports should be open, maybe even extra addresses should be added to the peerstore etc).

[probonopd]

Do I need to have a local ipfs daemon running in addition to my Go program? (I want to embed p2p file serving into my Go program, without the need for additional software to be installed on the user's machine.)