Similarities and comparison with LMQL, Guidance

[ogencoglu]

Thanks for the work!

A comparison table with LMQL and Guidance and similar libraries (e.g. like this) would be great. Any possibility for that?

[rlouf]

We have this figure in our paper:

What metrics would you find interesting?

[ogencoglu]

Similar features from the table would be nice. For example does it support token healing, json decoding, what types of constraints are available etc.

[Mihaiii]

Does outlines choses the top token (after considering the template constraints) from logits when sampling and therefore is only doing one inference call, even if the provided template combines standard text with generated text multiple times?

If I'm not mistaken, guidance will make a new inference call each time it hits a generative command within the template.

[wenxueru]

Does outlines support the token healing mentioned in Guidance?

[rlouf]

Soon :)

[freckletonj]

Here's my short summary from playing with these all recently:

• lmql: unusable morass of bugs
• guidance: best dev UI of all, truly great, but abandonware now
• outlines: no way to have a huge prompt with generations happening distributed throughout, and then named in a dictionary key to pull out later (like guidance)

[xiaohk]

You probably have seen this. guidance is now "reborn" 😆

[brandonwillard]

• outlines: no way to have a huge prompt with generations happening distributed throughout, and then named in a dictionary key to pull out later (like guidance)

Sounds like a good feature request! If you don't mind, create an issue and we can determine how it can meet these/your needs. We have a lot of big improvements and features in the pipeline, so it would be good to map this all out sooner than later.

[ogencoglu]

@freckletonj Would you like to elaborate on your lmql experience here or maybe over here ?

[brandonwillard]

For example does it support token healing, json decoding, what types of constraints are available etc.

Aside from the prompt construction features, outlines provides a very broad and run-time optimal approach to regex-guided generation, which also happens to cover JSON decoding. It also doesn't require token healing. We have a simple pydantic interface to the JSON decoding, too.

Presently, the library provides tools for the even broader task of (deterministic) CFG-guided generation and we plan to roll out a much more convenient interface for that feature soon. We have a highly efficient approach to this as well. See our technical paper for details.

More generally, outlines isn't trying to provide a single languages/DSL for the whole of constrained generation and/or prompting, and that might be one notable difference with lmql. We're currently focusing on composable, efficient core tools and features, which are then exposed though hackable integrations with established DSLs (e.g. pydantic for constraints, jinja2 for prompts) and some simple custom classes (e.g. Sequence for sampling). We have a lot of core design changes in the pipeline, but this mostly sums up the overall intent of what we've done and are doing.