List of resources, libraries and more for developers who would like to build with open-source machine learning off-the-shelf.
Motivation: Developers are often building machine learning with closed-source models behind gated APIs. These models can change by time without developers knowing, companies are giving away their data during inference and have no control over the model nor data.
There are a ton of open-source models out there that can be deployed by developers, but reducing barrier of entry to use these models and making developers aware of them are necessary, so I created this repository to do so.
Using the resources here, you can learn to find the model you need and serve it on the platform of your choice using the tools given here.
Hint: Take a look at foundation models section for one-model-fits-all type of models.
Note: To contribute, send a pull request to this repository. Note that this repository is focused on open-source machine learning.
- Resources
- Libraries, Platforms and Development Platform-specific Resources
- Modalities and Tasks
- Advanced
- Tasks: A documentation project to let developers build their first machine learning based product using models off-the-shelf.
- Open-source AI Cookbook: Recipes and notebooks using open-source models and libraries.
- Hugging Face Hub: Collaborative platform for machine learning. Discover hundreds of thousands of open-source models able to work off-the-shelf in /models.
- ONNX Runtime: Platform agnostic model runtime to use ML models.
- Transformers.js: A library to run cutting edge models directly in-browser.
- huggingface.js: A library to play with models on Hugging Face Hub through javascript.
- TensorFlow Lite: A library to deploy models on mobile and edge devices.
- Mediapipe: A framework that has prebuilt and customizable ML solutions, ready to deploy on Android, iOS.
- ExecuTorch: A library for enabling on-device ML in mobile/edge devices for PyTorch models.
- huggingface.dart: A Dart SDK to interact with the models on Hugging Face Hub.
- flutter-tflite: TensorFlow Lite Flutter plugin provides an easy, flexible, and fast Dart API to integrate TFLite models in flutter apps across mobile and desktop platforms.
- TensorFlow Lite: A library to deploy models on mobile and edge devices.
- ExecuTorch: A library for enabling on-device ML in mobile/edge devices for PyTorch models.
- Text Generation Inference: Toolkit to serve large language models.
- Text Embedding Inference: Toolkit to serve text embeddings.
- TorchServe: Flexible, easy to use and scalable inference server.
This section contains powerful models that can generalize well and can be used out-of-the-box.
The following resources are on zero-shot models: These models take in an image or text and possible classes in those images or texts.
- Zero-shot Object Detection
- Zero-shot Image Classification Resources
- Zero-shot Text Classification Resources
Note: The foundation model can be found under their associated task.
- Text Generation Inference: Toolkit to serve large language models.
- Text Embedding Inference: Toolkit to serve text embeddings.
- TGI Benchmark: Understanding throughput of inference servers, with TGI example.
- Open LLM Leaderboard: A leaderboard to find the open-source LLM for your use and budget.
- Awesome Open-source Foundation and Multimodal Models: A curated list of models that have high zero-shot capabilities and the models that can take in two different modalities (e.g. a model that can take in image and text and output text).
- Kosmos-2: Demo for Kosmos-2 model by Microsoft, that can understand image and text, answer questions about images, caption images, detect objects on images and gives answer without hallucinating.
- Fuyu-8B: Demo for Fuyu-8b by Adept, that can understand image and text and answer questions about images and caption images.
- Document AI Collection: A collection on demos and models for document AI.
- Stable Cascade: An app based on state-of-the-art image generation model Stable Cascade. (as of March '24)
- Stable Diffusion XL Inpainting: An application that can do inpainting when given a text prompt and an image.
- OWL: A curation about OWL model released by Google, the most powerful zero-shot object detection model. (as of March '24)
- Segment Anything: A curation about Segment Anything model released by Meta, the most powerful zero-shot image segmentation model. (as of March '24)
- Depth Anything: A highly practical solution for robust monocular depth estimation by training on a combination of 1.5M labeled images and 62M+ unlabeled images.
- Insanely Fast Whisper: A CLI to shrink and serve Whisper on-device.
- Open TTS Tracker: An awesome repository to keep a track of open-source text-to-speech models.
- Raycast Automate commands on macOS apps with a local ollama LLM, with Raycast extensions.