Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Helia providing content #657

Open
SgtPooki opened this issue Oct 17, 2024 · 4 comments
Open

docs: Helia providing content #657

SgtPooki opened this issue Oct 17, 2024 · 4 comments

Comments

@SgtPooki
Copy link
Member

We had a user reach out to us on slack about their content not being provided in the DHT. They were unaware that Helia does not provide content by default.

This is a different experience than Kubo, and we have no documentation mentioning this requirement.

What can we do:

  1. update docs to mention what expectations should be
  2. update Helia 101 example with providing steps
  3. Automatically provide when content is added
  4. Clarify when a user should understand "provide" to be completed
    • What does it mean that the promise from .provide resolves?
    • Should users be checking for how many FINAL_PEER events they get?
@SgtPooki
Copy link
Member Author

Seems related to libp2p/js-libp2p#2785

@gr33nT
Copy link

gr33nT commented Nov 12, 2024

Hey! I started building a project with Helia to learn about p2p and ipfs...

Is this still an issue?

My project is basically complete except for one problem:
Using the check tool, I can find my CID's if I also include the peer id of my helia node, but not with the CID alone..

  • Successfully connected to multiaddr
  • Found multiaddr advertised in the DHT
  • X Could not find the multihash in the DHT or IPNI. <- issue here
  • The peer responded it has the CID

I'm having a hard time finding any documentation on having a helia node advertise CIDs on the DHT.
If anyone is willing to point me in the right direction or show me how to do this, I would be very grateful.

@SgtPooki
Copy link
Member Author

@gr33nT Hey, did you provide your content explicitly? currently, Helia does not do this by default.

This issue is tracking the work required to document this more clearly so folks don't run into the issue you did :)

We are still not providing by default, and won't for the foresee-able future, but it's something we're open to discussing

@gr33nT
Copy link

gr33nT commented Nov 14, 2024

@SgtPooki Thanks for your response! So, I was NOT explicitly providing and that would explain everything 😅. Adding helia.routing.provide(fileCid); seems to work, i just have to straighten out my network config to be a little more reliable. Solving this has lead to a some more questions, however.

I would suspect the entry in DHT expires every so often and I would need to re-provide files?
Is this done automatically, or does re-providing also need to be explicit?
Is the information needed to do this persistent in the datastore?

also, if for whatever reason, I have files already in the blockstore but not provided, is there an easy way to loop through all the CID's and provide them again? Say, when the server starts?

I created my node like this

  const blockstore = new FsBlockstore(process.env[`BLOCKSTORE_PATH_${env}`]);
  const datastore = new FsDatastore(process.env[`DATASTORE_PATH_${env}`]);
  const agentVersion = `Helia ${libp2pInfo.name}/${libp2pInfo.version} UserAgent=${process.version}`;
  const libp2p = await createLibp2p({
    privateKey: libp2pKey,
    addresses: {
      listen: [
        `/ip4/0.0.0.0/tcp/${process.env[`LIBP2P_PORT_${env}`]}`,
        '/p2p-circuit',
        '/webrtc',
      ],
      announce: [
        `/ip4/${process.env[`LIBP2P_URL_${env}`]}/tcp/${process.env[`LIBP2P_PORT_${env}`]}`,
      ],
    },
    transports: [
      tcp(),
      circuitRelayTransport(),
      webRTC(),
      webRTCDirect(),
      webSockets(),
    ],
    connectionEncrypters: [
      noise(),
      tls(),
    ],
    streamMuxers: [
      yamux(),
    ],
    peerDiscovery: [
      mdns(),
      bootstrap({
        list: [
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN',
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb',
          '/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt',
          '/dnsaddr/va1.bootstrap.libp2p.io/p2p/12D3KooWKnDdG3iXw9eTFijk3EWSunZcFi54Zka4wmtqtt6rPxc8',
          '/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ',
        ],
      }),
    ],
    services: {
      autoNAT: autoNAT(),
      dcutr: dcutr(),
      aminoDHT: kadDHT({
        protocol: '/ipfs/kad/1.0.0',
        peerInfoMapper: removePrivateAddressesMapper,
        validators: {
          ipns: ipnsValidator,
        },
        selectors: {
          ipns: ipnsSelector,
        },
      }),
      identify: identify({
        agentVersion,
      }),
      identifyPush: identifyPush({
        agentVersion,
      }),
      ping: ping(),
      delegatedRouting: () => createDelegatedRoutingV1HttpApiClient('https://delegated-ipfs.dev/routing/v1'),
      relay: circuitRelayServer(),
      upnp: uPnPNAT(),
    },
  });

  return createHelia({
    datastore,
    blockstore,
    libp2p,
  });
}

const helia = await createNode();
console.log(helia.libp2p.getMultiaddrs());
const fs = unixfs(helia);

I've been adding content by doing:
const fileCid = await fs.addBytes(buf);
and now,
helia.routing.provide(fileCid);

I apologize if these questions seem basic or if Im missing something obvious in documentation. Any help or pointers would be greatly appreciated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants