Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic MacOS support #6

Merged
merged 10 commits into from
Jan 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion .github/workflows/rust-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ env:
CARGO_TERM_COLOR: always

jobs:
build:
build-linux:

name: Build on Linux
runs-on: ubuntu-latest

steps:
Expand All @@ -29,3 +30,25 @@ jobs:
- name: Test with cargo
run: |
cargo test

build-macos:

name: Build on MacOS
runs-on: macos-latest

steps:
- uses: actions/checkout@v4
- name: Prepare environment
run: |
brew install eccodes
rustup update stable
cargo clean
- name: Build with cargo
run: |
cargo build --release
- name: Check with clippy
run: |
cargo clippy -- -W clippy::pedantic
- name: Test with cargo
run: |
cargo test
25 changes: 25 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,28 @@ jobs:
run: |
cargo criterion
cargo clean

build-macos:

name: Build on MacOS
runs-on: macos-latest

steps:
- uses: actions/checkout@v4
- name: Prepare environment
run: |
brew install eccodes
rustup update stable
cargo clean
- name: Build with cargo
run: |
cargo build --release
cargo clean
- name: Test with cargo
run: |
cargo test
cargo clean
- name: Benchmark with criterion
run: |
cargo criterion
cargo clean
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
name = "eccodes"
description = "Unofficial high-level Rust bindings of the latest ecCodes release"
repository = "https://github.com/ScaleWeather/eccodes"
version = "0.7.0"
version = "0.8.0"
readme = "README.md"
authors = ["Jakub Lewandowski <[email protected]>"]
keywords = ["eccodes", "grib", "bufr", "meteorology", "weather"]
Expand Down
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ Read [this section](#crate-safety) to learn more about design decisions of this
open-source library for reading and writing GRIB and BUFR files
developed by [European Centre for Medium-Range Weather Forecasts](https://www.ecmwf.int/).

Because ecCodes supports mainly Linux platforms, this crate is not tested on other architectures.
This crate officially supports mainly Linux platforms as the ecCodes library supports them.
But it is possible to install ecCodes on MacOS and this crate successfully compiles and all tests pass.

## Usage

Expand All @@ -38,6 +39,12 @@ For example, on Ubuntu you can use `apt-get`:
sudo apt-get install libeccodes-dev
```

or `brew` on MacOS:

```bash
brew install eccodes
```

Alternatively, you can install the library manually from source in suitable directory
following [this instructions](https://confluence.ecmwf.int/display/ECC/ecCodes+installation).

Expand Down
13 changes: 8 additions & 5 deletions src/codes_handle/iterator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ use crate::{
///is corrupted and for some other reasons. The usage of `FallibleIterator` is sligthly different
///than usage of `Iterator`, check its documentation for more details.
///
///For a true memory safety and to provide a ful Rust Iterator functionality,
///this iterator clones each message to a new buffer.Although internal ecCodes
///message copy implementation makes this operation quite cheap, using this iterator
///For a true memory safety and to provide a ful Rust Iterator functionality,
///this iterator clones each message to a new buffer.Although internal ecCodes
///message copy implementation makes this operation quite cheap, using this iterator
///(and in effect this crate) comes with memory overhead, but is
///a necessity for memory safety.
///
Expand Down Expand Up @@ -188,8 +188,11 @@ mod tests {
// First, filter and collect the messages to get those that we want
let mut level: Vec<KeyedMessage> = handle
.filter(|msg| {
Ok(msg.read_key("shortName")?.value == KeyType::Str("msl".to_string())
&& msg.read_key("typeOfLevel")?.value == KeyType::Str("surface".to_string()))
Ok(
msg.read_key("shortName")?.value == KeyType::Str("msl".to_string())
&& msg.read_key("typeOfLevel")?.value
== KeyType::Str("surface".to_string()),
)
})
.collect()
.unwrap();
Expand Down
64 changes: 34 additions & 30 deletions src/codes_handle/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

use crate::errors::CodesError;
use bytes::Bytes;
use eccodes_sys::{ProductKind_PRODUCT_GRIB, codes_handle, codes_keys_iterator, codes_nearest};
use eccodes_sys::{codes_handle, codes_keys_iterator, codes_nearest, ProductKind_PRODUCT_GRIB};
use errno::errno;
use libc::{c_char, c_void, size_t, FILE};
use log::warn;
Expand Down Expand Up @@ -131,7 +131,7 @@ pub struct NearestGridpoint {
///Distance from coordinates requested in `find_nearest()`
pub distance: f64,
///Value of the filed at given coordinate
pub value: f64,
pub value: f64,
}

impl CodesHandle {
Expand Down Expand Up @@ -314,8 +314,8 @@ mod tests {
use eccodes_sys::ProductKind_PRODUCT_GRIB;

use crate::codes_handle::{CodesHandle, DataContainer, ProductKind};
use std::path::Path;
use log::Level;
use std::path::Path;

#[test]
fn file_constructor() {
Expand Down Expand Up @@ -363,35 +363,39 @@ mod tests {
async fn codes_handle_drop() {
testing_logger::setup();

let file_path = Path::new("./data/iceland-surface.grib");
let product_kind = ProductKind::GRIB;

let handle = CodesHandle::new_from_file(file_path, product_kind).unwrap();
drop(handle);
{
let file_path = Path::new("./data/iceland-surface.grib");
let product_kind = ProductKind::GRIB;

testing_logger::validate(|captured_logs| {
assert_eq!(captured_logs.len(), 0);
});
let handle = CodesHandle::new_from_file(file_path, product_kind).unwrap();
drop(handle);

let product_kind = ProductKind::GRIB;
let file_data = reqwest::get(
"https://github.com/ScaleWeather/eccodes/blob/main/data/iceland.grib?raw=true",
)
.await
.unwrap()
.bytes()
.await
.unwrap();
testing_logger::validate(|captured_logs| {
assert_eq!(captured_logs.len(), 0);
});
}

let handle = CodesHandle::new_from_memory(file_data, product_kind).unwrap();
drop(handle);

//logs from Reqwest are expected
testing_logger::validate(|captured_logs| {
for log in captured_logs {
assert_ne!(log.level, Level::Warn);
assert_ne!(log.level, Level::Error);
}
});
{
let product_kind = ProductKind::GRIB;
let file_data = reqwest::get(
"https://github.com/ScaleWeather/eccodes/blob/main/data/iceland.grib?raw=true",
)
.await
.unwrap()
.bytes()
.await
.unwrap();

let handle = CodesHandle::new_from_memory(file_data, product_kind).unwrap();
drop(handle);

//logs from Reqwest are expected
testing_logger::validate(|captured_logs| {
for log in captured_logs {
assert_ne!(log.level, Level::Warn);
assert_ne!(log.level, Level::Error);
}
});
}
}
}
24 changes: 18 additions & 6 deletions src/intermediate_bindings.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
#![allow(non_camel_case_types)]

//!Module containing intermediate (type) bindings to ecCodes functions.
//!
//!These bindings convert Rust types to correct C types
Expand All @@ -10,11 +12,17 @@ use std::{
ptr::{self, addr_of_mut},
};

#[cfg(target_os = "macos")]
type _SYS_IO_FILE = eccodes_sys::__sFILE;

#[cfg(not(target_os = "macos"))]
type _SYS_IO_FILE = eccodes_sys::_IO_FILE;

use eccodes_sys::{
codes_context, codes_handle, codes_keys_iterator, codes_nearest, CODES_NEAREST_SAME_DATA,
CODES_NEAREST_SAME_GRID, CODES_TYPE_BYTES, CODES_TYPE_DOUBLE, CODES_TYPE_LABEL,
CODES_TYPE_LONG, CODES_TYPE_MISSING, CODES_TYPE_SECTION, CODES_TYPE_STRING,
CODES_TYPE_UNDEFINED, _IO_FILE,
CODES_TYPE_UNDEFINED,
};
use libc::{c_void, FILE};
use num_traits::FromPrimitive;
Expand Down Expand Up @@ -46,7 +54,7 @@ pub unsafe fn codes_handle_new_from_file(

let file_handle = eccodes_sys::codes_handle_new_from_file(
context,
file_pointer.cast::<_IO_FILE>(),
file_pointer.cast::<_SYS_IO_FILE>(),
product_kind as u32,
&mut error_code,
);
Expand Down Expand Up @@ -275,9 +283,11 @@ pub unsafe fn codes_get_message(
return Err(err.into());
}

assert!(buffer_size == message_size,
assert!(
buffer_size == message_size,
"Buffer and message sizes ar not equal in codes_get_message!
Please report this panic on Github.");
Please report this panic on Github."
);

Ok((buffer_ptr, message_size))
}
Expand All @@ -300,9 +310,11 @@ pub unsafe fn codes_get_message_copy(handle: *mut codes_handle) -> Result<Vec<u8
return Err(err.into());
}

assert!((buffer_size == message_size && message_size == buffer.len()),
assert!(
(buffer_size == message_size && message_size == buffer.len()),
"Buffer, vector and message sizes ar not equal in codes_get_message!
Please report this panic on Github.");
Please report this panic on Github."
);

Ok(buffer)
}
Expand Down
27 changes: 18 additions & 9 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
//!Because of the ecCodes library API characteristics theses bindings are
//!rather thick wrapper to make this crate safe and convenient to use.
//!
//!Because ecCodes supports mainly Linux platforms, this crate is not tested on other architectures.
//!This crate officially supports mainly Linux platforms as the ecCodes library supports them.
//!But it is possible to install ecCodes on MacOS and this crate successfully compiles and all tests pass.
//!
//!If you want to see more features released quicker do not hesitate
//!to contribute and check out [Github repository](https://github.com/ScaleWeather/eccodes).
Expand Down Expand Up @@ -48,7 +49,7 @@
//!
//!```
//!// We are reading the mean sea level pressure for 4 gridpoints
//!// nearest to Reykjavik (64.13N, -21.89E) for 1st June 2021 00:00 UTC
//!// nearest to Reykjavik (64.13N, -21.89E) for 1st June 2021 00:00 UTC
//!// from ERA5 Climate Reanalysis
//!
//!// Open the GRIB file and create the CodesHandle
Expand Down Expand Up @@ -82,17 +83,17 @@
//!let nearest_gridpoints = level.find_nearest(64.13, -21.89)?;
//!
//!// Print value and distance of the nearest gridpoint
//!println!("value: {}, distance: {}",
//! nearest_gridpoints[3].value,
//!println!("value: {}, distance: {}",
//! nearest_gridpoints[3].value,
//! nearest_gridpoints[3].distance);
//!# Ok(())
//!# }
//!```
//!
//!### Writing GRIB files
//!
//!The crate provides a basic support for setting `KeyedMessage` keys
//!and writing GRIB files. The easiests (and safest) way to create a
//!The crate provides a basic support for setting `KeyedMessage` keys
//!and writing GRIB files. The easiests (and safest) way to create a
//!new custom message is to copy exisitng one from other GRIB file,
//!modify the keys and write to new file.
//!
Expand Down Expand Up @@ -180,7 +181,13 @@
//!For example, on Ubuntu you can use `apt-get`:
//!
//!```text
//!$ sudo apt-get install libeccodes-dev
//!sudo apt-get install libeccodes-dev
//!```
//!
//!or `brew` on MacOS:
//!
//!```text
//!brew install eccodes
//!```
//!
//!Alternatively, you can install the library manually from source in suitable directory
Expand Down Expand Up @@ -213,6 +220,8 @@ pub mod codes_handle;
pub mod errors;
mod intermediate_bindings;

pub use fallible_iterator::{FallibleIterator, IntoFallibleIterator};
pub use codes_handle::{CodesHandle, Key, KeyedMessage, KeyType, KeysIteratorFlags, NearestGridpoint, ProductKind};
pub use codes_handle::{
CodesHandle, Key, KeyType, KeyedMessage, KeysIteratorFlags, NearestGridpoint, ProductKind,
};
pub use errors::CodesError;
pub use fallible_iterator::{FallibleIterator, IntoFallibleIterator};