docs: add blog posts for npm-on-ipfs #215
Conversation
ghost
added
the
|
Looking good 👍 Can users set up their own hosted mirrors using the same code? If so, that would be something worth mentioning at the end of the post as well. |
| @@ -0,0 +1,99 @@ | |||
| --- | |||
| date: 2019-02-17 | |||
| title: Putting npm on IPFS Part 1 - The Registry | |||
There was a problem hiding this comment.
I like the structure, separating this into the server/client posts definitely makes it more digestible, and the breakout "quick how to" at the top of each is good 👍
|
|
||
|  | ||
|
|
||
| If this sounds familiar it may be because you've just read [part 1](/post/70-putting-npm-on-ipfs-part-1) - indeed it's the exact same code fulfilling the exact same function, just locally on your machine instead of on a server somewhere. |
There was a problem hiding this comment.
this bit papers over rather a lot of functionality -- stuff like persistent/proc/external node, whether the NPM update log is replicated, etc (I think? this is based on my reading of the code)
maybe something to unpack a bit more? (it's sort of addressed in the docs linked from "other configurations are available" above, but that's not abundantly clear)
There was a problem hiding this comment.
I've expanded the part on how it works, let me know if you'd like more added.
|
|
||
| The http server is necessary because at the time of writing neither npm or Yarn support IPFS as a transport - hopefully one day this won't be necessary but as it stands no http traffic leaves your machine, unless you request modules that don't have CIDs in their metadata. | ||
|
|
||
| It also dials the ipfs-npm-registry-mirror directly to improve the speed of content resolution - this is partly because we know that the mirror will have the content we are after but also because there is no [DHT](https://ipfs.io/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/Distributed_hash_table.html) in js-IPFS (yet - it's coming!) so discovery would have to occur via the gateway nodes. |
There was a problem hiding this comment.
isn't DHT available as of v0.35.0, or am I misreading the release notes?
it might be worth mentioning that in the future this is something that could be handled by rendezvous, as well
There was a problem hiding this comment.
@parkan I don't think v0.35.0 final has been published to npm yet
There was a problem hiding this comment.
Quite right. It may not be enabled by default until v0.36.0 either.
|
|
||
| ## 🎁 What's next? | ||
|
|
||
| So many things! |
There was a problem hiding this comment.
seems like a perfect opportunity to tease future package manager work -- probably don't want to go into too much specifics here, but can definitely do a bit more!
There was a problem hiding this comment.
+1! Maybe it's time to create a public-facing package managers working group for IPFS that we can point people to for things like upcoming roadmap, categories, synthesized research, next steps, etc!
There was a problem hiding this comment.
I put a few things down here but only wanted to mention npm-related things as that's the focus of the blog post. If there's anything you want added please let me know.
|
overall this is good! let me know if you want me to to a quick style pass before publishing 🙂 |
There was a problem hiding this comment.
This is looking really good and I like the two part approach. I recommend posting them at 2 different dates.
Right now what I see missing is "painting the picture", the posts require a lot of text and knowing how infra works to be able to imagine how the things are set up. Let's make it easy for folks to understand.
| ** drumroll ** | ||
|
|
||
| ## 🙌 Introducing ipfs-npm | ||
|
|
There was a problem hiding this comment.
Needs diagram of what this means :)
There was a problem hiding this comment.
I've reused the graphic from the npm-on-ipfs readme as I'm not sure what a diagram here would look like - what did you have in mind?
@andrew done!
@parkan Please use the "Suggested change" function for any improvements you have and/or otherwise leave a comment |
|
theoretical performance improvements are great, but if we make those claims we should back them up with some benchmarks. I found this little benchmark suite that someone used to compare a few installers and quickly modified it to include ipfs-npm: https://github.com/parkan/npm-cache-benchmark unfortunately I'm having some connection issues right now so I can't even make it through the benchmarks, can someone else run this and see how we compare? (would need to do a separate run with a clean repo to establish a baseline) |
|
I spent a couple hours today trying to make things work well enough to be testable but was not successful: ipfs-shipyard/npm-on-ipfs#89 my suggestion would be:
@achingbrain thank you for putting up with these revisions! |
|
let's circle back and try to publish this next week |
|
Made a few suggested edits based on Arkadiy's feedback. I think we should aim to push this live ASAP - there's some npm kerfuffle this week and would be nice to attract interested parties to this distributed npm experiment while that interest and engagement is fresh. |
| ... | ||
| ``` | ||
|
|
||
| The CID resolves to the tarball for a module, but while the module has been added to IPFS is not stored in the MFS so we resolve it using the CID: |
There was a problem hiding this comment.
| The CID resolves to the tarball for a module, but while the module has been added to IPFS is not stored in the MFS so we resolve it using the CID: | |
| The CID resolves to the tarball for a module, but although the module has been added to IPFS is not stored in the MFS so we resolve it using the CID: |
Explain why it's not necessary to store it in MFS.
There was a problem hiding this comment.
why is it not necessary to store it in MFS?
| The CID resolves to the tarball for a module, but while the module has been added to IPFS is not stored in the MFS so we resolve it using the CID: | ||
|
|
||
| ```console | ||
| $ jsipfs files read /ipfs/bafybeiafts7s65iodk4wsetucwhury3cpk4fge374wxhdu5vzc4zuli4xi |
There was a problem hiding this comment.
You've just said it's not stored in MFS but have used a command we've documented as being for primary use with MFS to read it - perhaps using ipfs cat is more appropriate here?
|
|
||
|  | ||
|
|
||
| When it sees a new module published, the replicator pulls down any new tarballs, adds them to IPFS and updates the metadata with the CIDs of the new module. |
There was a problem hiding this comment.
...and explain that this is the reason we're using MFS?
|
|
||
| What if you could skip that part and just use IPFS? | ||
|
|
||
| Stay tuned for [Putting npm on IPFS Part 2 - The Client](/post/74-putting-npm-on-ipfs-part-2) |
There was a problem hiding this comment.
Can't wait. Can we get this one merged and published 🚢🚢🚢??
There was a problem hiding this comment.
@achingbrain can has final review and finishing touches pls?
|
Can someone write a list of action items left to ship this? I can't tell if everything has been addressed or not :) |
|
Perhaps this could be scheduled along side the launch of ipfs-desktop support for npm-on-ipfs? |
|
@andrew I would rather not wait any longer. These posts have so much valuable information and that has been trapped in this PR forever. |
|
@daviddias Lets not relase this just yet. The registry mirror is unreliable at the moment. There is lots of good stuff in this blog post, but I don't feel we are ready to encourage people to use it yet. Alex is getting access to the infra to fix that, I am digging into the UX issues with @jessicaschilling and @andrew and we are working on a plan. |
|
@olizilla Ack! Thanks for giving me the update. Are there issues to track that work? Would you like to update your comment to point to those? |
|
Here is a first draft of a release issue for npm-on-ipfs: desktop edition |
|
@olizilla the npm-on-ipfs: desktop edition does look very exciting! One of the original intents of writing these blog posts was to have a good resource to show to the public that there is indeed a way to use S3 or other storage backend and that there is indeed a way to store large amounts of data onto IPFS. Let's not forget this focus while revamping the communication to be the "Desktop version" |
|
Now that IPFS Desktop has npm-on-ipfs as an experiment, how do we feel about resurrecting these posts? I think the explanation work here is awesome and want to see it get the reach it deserves. @olizilla @achingbrain @hacdias - do you have ideas? |
|
@momack2 overall, I think it'd be a great idea to actually post this |
|
Closing: stale PR, and migration to new https://github.com/ipfs/ipfs-blog nearly complete. |
PR'd in from a fork because I can't push branches to this repo.
This adds two new blog posts on IPFS and npm.
ipfs-npmto cut out the http/dns middlemanThis is the first blog post(s) I've written so hopefully I've put all the things in the right places.
@jacobheun The section on
datastore-s3is a bit bare, is there anything else you'd add?@andrew Could you give it a proof-read please?