Tracking: Collaboration with other Node.js Micropub projects

[voxpelli]

This is a tracking issue for collaboration work in my part of the Node.js IndieWeb ecosystem.

Issues in other repos

• Collaboration on parts of the code? getindiekit/indiekit#1
• Collaboration on parts of the code? vipickering/mastr-cntrl#64
• Collaboration on parts of the code? grantcodes/postr#43

My Micropub-related modules

• micropub-express – an Express Micropub endpoint that accepts and verifies Micropub requests and calls a callback with a parsed micropubDocument.
• format-microformat – a module that takes a micropubDocument as its input, and converts this data into a standard that can be published elsewhere. Currently supports the Jekyll format.
• github-publish – a module that takes a filename and content and publishes it to a GitHub repository. The formatted data generated by format-microformat can be published to a Jekyll blog hosted on a GitHub, or a GitHub Pages site.

Other IndieWeb projects of mine

• webmention.herokuapp.com – an open source hosted Webmention endpoint

Collaboration opportunities found so far

• Deletion support in node-github-publish: Also support deletions node-github-publish#34
• (...)

[grantcodes]

Excellent idea! I'm all for it!

I don't know if you've mentioned them already but I've talked with @EdwardHinkle and @martymcguire about a similar idea of having express middleware to do generic things.

I'm also the proud owner of the @indieweb organization on mom if we ever get to a stage where we have something were happy to publish as a group.

As for relation to my current personal projects, I have a few endpoints I have made - mainly a micropub endpoint, webmention endpoint and token endpoint. The majority of the code it fairly resuable but not 100% designed as npm modules. The goal for my project is to be a fairly feature complete, fully extendable indieweb backend with the frontend totally independent. I would love if it was usable with static sites but that is not my main goal.

In my indieweb dreams I would love to have a bunch of reusable endpoints / modules to cover a range on indieweb functionality. Mainly from the top of my head:

• micropub
• tokens
• auth
• webmention accept
• webmention send (I think there is already a good implementation of this)
• post type discovery
• format conversion (mf2 to jf2 to md frontmatter etc)
• maybe url parsing (basically mf2 html parser + other metadata fallback)

That's about all I can think of for now on my phone but I'm sure there is much more

[grantcodes]

Forgot media endpoint

[grantcodes]

Micropub Endpoints

So I've done a quick look into the various js based micropub endpoints and here is what I have found them to support based on micropub.rocks tests. (note this absolutely may be missing projects or features of each project)

Test Number	Test Name	grantcodes/micropub-endpoint	voxpelli/node-micropub-express	paulrobertlloyd/indiekit	muan/micropub-endpoint
100	Create an h-entry post (form-encoded)	✔	✔	✔	✔
101	Create an h-entry post with multiple categories (form-encoded)	✔	✔	✔	✔
104	Create an h-entry with a photo referenced by URL (form-encoded)	✔	✔	❌	✔
107	Create an h-entry post with one category (form-encoded)	✔	✔	✔	✔
200	Create an h-entry post (JSON)	✔	✔	✔	❌
201	Create an h-entry post with multiple categories (JSON)	✔	✔	✔	❌
202	Create an h-entry with HTML content (JSON)	✔	✔	✔	❌
203	Create an h-entry with a photo referenced by URL (JSON)	✔	✔	✔	❌
204	Create an h-entry post with a nested object (JSON)	✔	✔	✔	❌
205	Create an h-entry post with a photo with alt text (JSON)	✔	✔	✔	❌
206	Create an h-entry with multiple photos referenced by URL (JSON)	✔	✔	✔	❌
300	Create an h-entry with a photo (multipart)	✔	✔	❌	✔
301	Create an h-entry with two photos (multipart)	✔	✔	❌	✔
400	Replace a property	✔	❌	❌	✔
401	Add a value to an existing property	✔	❌	❌	❌
402	Add a value to a non-existent property	✔	❌	❌	❌
403	Remove a value from a property	✔	❌	❌	❌
404	Remove a property	✔	❌	❌	❌
405	Reject the request if operation is not an array	✔	❌	❌	✔
500	Delete a post (form-encoded)	✔	❌	✔	✔
501	Delete a post (JSON)	✔	❌	✔	✔
502	Undelete a post (form-encoded)	✔	❌	❌	❌
503	Undelete a post (JSON)	✔	❌	❌	❌
600	Configuration Query	✔	❓	✔	✔
601	Syndication Endpoint Query	✔	❓	✔	❌
602	Source Query (All Properties)	✔	❓	✔	❌
603	Source Query (Specific Properties)	✔	❓	✔	❌
700	Upload a jpg to the Media Endpoint	✔	❌	✔	✔
701	Upload a png to the Media Endpoint	✔	❌	✔	✔
702	Upload a gif to the Media Endpoint	✔	❌	✔	✔
800	Accept access token in HTTP header	✔	✔	✔	✔
801	Accept access token in POST body	✔	✔	✔	✔
802	Does not store access token property	✔	✔	✔	✔
803	Rejects unauthenticated requests	✔	✔	✔	✔
804	Rejects unauthorized access tokens	✔	✔	✔	✔

Other projects I looked at:

• EdwardHinkle/abode but looks like it is based on voxpelli/node-micropub-express
• astillanet/micropub-node-parser - very very basic
• sa-mm/micropub_server - Appears to use voxpelli/node-micropub-express with graphql
• joshdick/microstat - Uses voxpelli/node-micropub-express and voxpelli/node-format-microformat to create markdown files
• am1t/blotpub - Uses a fork of voxpelli/node-micropub-express to talk to dropbox or blot.im
• vipickering/mastr-cntrl - Looks very constrained per post type

#humblebrag - definitely looks like my own endpoint is the most feature complete, but that definitely doesn't make it the best option.

[EdwardHinkle]

I think this is a great effort. Mine (abode) is currently using voxpelli/Node-Micropub-express as grant mentioned. I have been planning on building one from scratch so this effort would definitely help on preventing me from re-inventing the wheel. A couple important aspects in my mind:

• the output of the Middleware Library shouldn’t try to do anything fancy. It should output standard MF2 JSON and then individual libraries that use the Micropub Endpoint base can convert the mf2 JSON to whatever is needed.

• support for Update and Delete are definitely essential for me.

Some other thoughts:

• it would be useful if we have the Micropub Endpoint take either a Token Endpoint Url OR a function. This would allow for hard-coding tokens or other internal integrations.

Ultimately though, I think it would be amazing if we were able to all contribute to Node.js libraries in npm that allowed for customizable/extensionable integration into other apps.

Also I don’t know how much of you all use TypeScript but adding TypeScript support to them as well would be pretty great.

[grantcodes]

I agree with @EdwardHinkle

The endpoint should definitely just pass through mf2 json, and should at least be able to handle all of those tests from micropub.rocks I listed above.

Another couple of notes on what a shared endpoint could potentially handle:

• It should be able to handle scopes and reject if not sufficient
• Should validate mf2 and reject if invalid (just basic stuff like properties should be arrays)
• Handle media. This is a bit more complicated, but I think this should also provide a media endpoint that can be set to just store to a folder, but then also be extendable with a function to handle files or pass them to an external media endpoint

The only thing I am not sure I agree with is TypeScript, I think TypeScript is great but if it is a community project it is just one extra thing that someone would have to learn before being able to contribute.

[grantcodes]

Another thing I'd like to see decided prior to starting any development would be about the actual method of using the library.

For now voxpelli/node-micropub-express uses a object to configure the endpoint:

const micropub = require('micropub-express');

app.use('/micropub', micropub({
  tokenReference: {
    me: 'http://example.com/',
    endpoint: 'https://tokens.indieauth.com/token',
  },
  handler: async (micropubDocument, req) => ({ url: 'http://example.com/url/to/new/post' })
}));

But the other options is to use methods, more like how express actually works:

const Micropub = require('micropub-express');
const micropub = Micropub();

micropub.handler(async (micropubDocument, req) => ({ url: 'http://example.com/url/to/new/post' });

micropub.setTokenReference({
  me: 'http://example.com/',
  endpoint: 'https://tokens.indieauth.com/token',
});

app.use('/micropub', micropub.router);

Note: just examples, not suggesting any sort of vocabulary.

Personally I think I prefer using the methods, but I am not 100% sure of the pros and cons

[voxpelli]

I think I opted for using an object to configure as then it’s easier to validate early on that all necessary config has been given, but I’m neither very opinionated about it tors 28 feb. 2019 kl. 13:31 skrev Grant Richmond <[email protected]>:

…

Another thing I'd like to see decided prior to starting any development would be about the actual method of using the library. For now voxpelli/node-micropub-express uses a object to configure the endpoint: const micropub = require('micropub-express'); app.use('/micropub', micropub({ tokenReference: { me: 'http://example.com/', endpoint: 'https://tokens.indieauth.com/token', }, handler: async (micropubDocument, req) => ({ url: 'http://example.com/url/to/new/post' }) })); But the other options is to use methods, more like how express actually works: const Micropub = require('micropub-express');const micropub = Micropub(); micropub.handler((micropubDocument, req) => ({ url: 'http://example.com/url/to/new/post' }); micropub.setTokenReference({ me: 'http://example.com/', endpoint: 'https://tokens.indieauth.com/token', }); app.use('/micropub', micropub.router); Note: just examples, not suggesting any sort of vocabulary. Personally I think I prefer using the methods, but I am not 100% sure of the pros and cons — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#97 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACGmWW0kYdbEd1-pqrS_KqVOTrBTksRks5vR8wMgaJpZM4bKIdy> .

[miklb]

happy summer friends. Wondering if there's been further discussion on this and getindiekit/indiekit#1 (comment)

I'm eerily close to moving back to Jekyll and assessing the landscape. I'm a big fan of everyone's work.

[grantcodes]

@miklb I don't think there's been much extra discussion elsewhere yet. But I'd be keen to get started on something soon. Particularly on micropub, token and auth endpoints.

I think there are already at least a couple of express token endpoints out there that are pretty much done.

For micropub the projects mentioned above are all good, but handle things differently, but a new higher level module could probably be put together using some of the best parts from each.

As for auth, I don't think there is much built so far that would be easily reusable.

But I am not really sure what would be a best starting point. Maybe a new repo with a project that lays out a bit of a road map?

[grantcodes]

I went ahead and made a new repo specifically for micropub endpoint planning: https://github.com/grantcodes/future-micropub-endpoint

[paulrobertlloyd]

@miklb You asked this question at just the right time; I had not been working on IndieKit since March, but started looking at it again this month, so it’s very much front of mind.

I’m excited by the moves @grantcodes has made to elicit collaboration among different project authors, and will start responding with my thoughts and contributions at the above repo.

[miklb]

@paulrobertlloyd I've been lurking watching your repo so can't claim serendipity :-)

I'm excited too and am a willing guinea pig for my new site. Certainly will try to chip in where I can code wise or other types of contribution.