Skip to content

Commit

Permalink
[stacked] Add docs to inventory code (#864)
Browse files Browse the repository at this point in the history
* Add module docs and example

* Document resolve and partial_resolve

* Document more inventory methods

* Document Artifact

* Document ArtifactRequirement

I also suggest we change the name of `inventory/version` to `inventory/artifact_requirement.rs` or `requirement.rs`.

* Update example to compare Checksum instead of string

* Apply suggestions from code review

Co-authored-by: Rune Soerensen <[email protected]>

* Update feature names

* Rewrite example usage

* Show how to display checksum in example

---------

Co-authored-by: Rune Soerensen <[email protected]>
  • Loading branch information
schneems and runesoerensen authored Oct 3, 2024
1 parent 453861c commit 1ac0955
Show file tree
Hide file tree
Showing 3 changed files with 123 additions and 0 deletions.
106 changes: 106 additions & 0 deletions libherokubuildpack/src/inventory.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,94 @@
//! # Inventory
//!
//! Many buildpacks need to provide artifacts from different URLs. A helpful pattern
//! is to provide a list of artifacts in a TOML file, which can be parsed and used by
//! the buildpack to download the correct artifact. For example, a Ruby buildpack
//! might need to download pre-compiled Ruby binaries hosted on S3.
//!
//! This module can be used to produce and consume such an inventory file.
//!
//! ## Features
//!
//! - Version lookup and comparison: To implement the inventory, you'll need to define how
//! versions are compared. This allows the inventory code to find an appropriate artifact
//! based on whatever custom version logic you need. If you don't need custom logic, you can
//! use the included `inventory-semver` feature.
//! - Architecture aware: Beyond version specifiers, buildpack authors may need to provide different
//! artifacts for different computer architectures such as ARM64 or AMD64. The inventory encodes
//! this information which is used to select the correct artifact.
//! - Checksum validation: In addition to knowing the URL of an artifact, buildp authors
//! want to be confident that the artifact they download is the correct one. To accomplish this
//! the inventory contains a checksum of the download and can be used to validate the download
//! has not been modified or tampered with. To use sha256 or sha512 checksums out of the box,
//! enable the `inventory-sha2` feature
//! - Extensible with metadata: The default inventory format covers a lot of common use cases,
//! but if you need more, you can extend it by adding custom metadata to each artifact.
//!
//! ## Example usage
//!
//! This example demonstrates:
//! * Creating an artifact using the `inventory-sha2` and `inventory-semver` features.
//! * Adding the artifact to an inventory.
//! * Serializing and deserializing the inventory [to](Inventory#method.fmt) and [from](Inventory::from_str) TOML.
//! * [Resolving an inventory artifact](Inventory::resolve) specifying relevant OS, architecture, and version requirements.
//! * Using the resolved artifact's checksum value to verify "downloaded" data.
//!
//! ```rust
//! use libherokubuildpack::inventory::{artifact::{Arch, Artifact, Os}, Inventory, checksum::Checksum};
//! use semver::{Version, VersionReq};
//! use sha2::{Sha256, Digest};
//!
//! // Create an artifact with a SHA256 checksum and `semver::Version`
//! let new_artifact = Artifact {
//! version: Version::new(1, 0, 0),
//! os: Os::Linux,
//! arch: Arch::Arm64,
//! url: "https://example.com/foo.txt".to_string(),
//! checksum: "sha256:2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"
//! .parse::<Checksum<Sha256>>()
//! .unwrap(),
//! metadata: None,
//! };
//!
//! // Create an inventory and add the artifact
//! let mut inventory = Inventory::<Version, Sha256, Option<()>>::new();
//! inventory.push(new_artifact.clone());
//!
//! // Serialize the inventory to TOML
//! let inventory_toml = inventory.to_string();
//! assert_eq!(
//! r#"[[artifacts]]
//! version = "1.0.0"
//! os = "linux"
//! arch = "arm64"
//! url = "https://example.com/foo.txt"
//! checksum = "sha256:2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"
//! "#,
//! inventory_toml
//! );
//!
//! // Deserialize the inventory from TOML
//! let parsed_inventory = inventory_toml
//! .parse::<Inventory<Version, Sha256, Option<()>>>()
//! .unwrap();
//!
//! // Resolve the artifact by OS, architecture, and version requirement
//! let version_req = VersionReq::parse("=1.0.0").unwrap();
//! let resolved_artifact = parsed_inventory.resolve(Os::Linux, Arch::Arm64, &version_req).unwrap();
//!
//! assert_eq!(&new_artifact, resolved_artifact);
//!
//! // Verify checksum of the resolved artifact
//! let downloaded_data = "foo"; // Example downloaded file content
//! let downloaded_checksum = Sha256::digest(downloaded_data).to_vec();
//!
//! assert_eq!(downloaded_checksum, resolved_artifact.checksum.value);
//! println!(
//! "Successfully downloaded {} with checksum {}",
//! resolved_artifact.url,
//! hex::encode(&resolved_artifact.checksum.value)
//! );
//! ```
pub mod artifact;
pub mod checksum;
pub mod version;
Expand All @@ -18,6 +109,12 @@ use std::fmt::Formatter;
use std::str::FromStr;

/// Represents an inventory of artifacts.
///
/// An inventory can be read directly from a TOML file on disk and used by a buildpack to resolve
/// requirements for a specific artifact to download.
///
/// The inventory can be manipulated in-memory and then re-serialized to disk to facilitate both
/// reading and writing inventory files.
#[derive(Debug, Serialize, Deserialize)]
pub struct Inventory<V, D, M> {
#[serde(bound = "V: Serialize + DeserializeOwned, D: Digest, M: Serialize + DeserializeOwned")]
Expand All @@ -31,15 +128,20 @@ impl<V, D, M> Default for Inventory<V, D, M> {
}

impl<V, D, M> Inventory<V, D, M> {
/// Creates a new empty inventory
#[must_use]
pub fn new() -> Self {
Self::default()
}

/// Add a new artifact to the in-memory inventory
pub fn push(&mut self, artifact: Artifact<V, D, M>) {
self.artifacts.push(artifact);
}

/// Return a single artifact as the best match given the input constraints
///
/// If multiple artifacts match the constraints, the one with the highest version is returned.
pub fn resolve<R>(&self, os: Os, arch: Arch, requirement: &R) -> Option<&Artifact<V, D, M>>
where
V: Ord,
Expand All @@ -56,6 +158,10 @@ impl<V, D, M> Inventory<V, D, M> {
.max_by_key(|artifact| &artifact.version)
}

/// Resolve logic for Artifacts that implement `PartialOrd` rather than `Ord`
///
/// Some version implementations are only partially ordered. One example could be f32 which is not totally ordered
/// because NaN is not comparable to any other number.
pub fn partial_resolve<R>(
&self,
os: Os,
Expand Down
9 changes: 9 additions & 0 deletions libherokubuildpack/src/inventory/artifact.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,15 @@ use serde::{Deserialize, Serialize};
use std::fmt::Display;
use std::str::FromStr;

/// Representation of a downloadable artifact such as a binary tarball.
///
/// An inventory is made up of multiple artifacts that have a version that
/// can be compared to each other and a URL where the artifact can be downloaded.
///
/// Artifacts are OS and architectures specific. The checksum value can
/// be used to validate an artifact once it has been downloaded.
///
/// Metadata can be used to store additional information about the artifact.
#[derive(Debug, Serialize, Deserialize, Clone)]
pub struct Artifact<V, D, M> {
#[serde(bound = "V: Serialize + DeserializeOwned")]
Expand Down
8 changes: 8 additions & 0 deletions libherokubuildpack/src/inventory/version.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,17 @@
/// Represents the requirements for a valid artifact
///
/// Checks the version and metadata of an artifact are valid or not
pub trait ArtifactRequirement<V, M> {
/// Return true if the given metadata satisfies the requirement
fn satisfies_metadata(&self, metadata: &M) -> bool;

/// Return true if the given version satisfies the requirement
fn satisfies_version(&self, version: &V) -> bool;
}

/// Check if the version satisfies the requirement (ignores Metadata)
pub trait VersionRequirement<V> {
/// Return true if the given version satisfies the requirement
fn satisfies(&self, version: &V) -> bool;
}

Expand Down

0 comments on commit 1ac0955

Please sign in to comment.