Ruby

[ggerganov]

General discussion about the ruby bindings for whisper.cpp

Current state: bindings/ruby

Initial merge: #500
CC: @taf2

[taf2]

Working on getting a gem published to rubygems so people can just

gem install whispercpp

[taf2]

Looks like someone already published a gem named whisper... So I think if we go with whispercpp - that would make sense and is available. Will work on pushing a gem and pull request.

[taf2]

@ggerganov I'm looking for good solution to interrupt whisper_full_parallel. In ruby we can wrap the call to this in an unlock method, since ruby has a global interpreter lock it is ideal to unlock while the c++ code runs allowing ruby to run other threads and even be interrupted via ctrl+c and other interrupt signals. It looks whisper_full_parallel calls whisper_full in each thread or just calls whisper_full, if there was a maybe volatile stop variable we added to the context then we could toggle that to false/0 and cause the while loop to break. thoughts?

[ggerganov]

There is currently an existing abort mechanism via the "encoder_begin" callback:

whisper.cpp/whisper.h

Lines 240 to 245 in f420de1

	// Encoder begin callback
	// If not NULL, called before the encoder starts
	// If it returns false, the computation is aborted
	typedef bool (*whisper_encoder_begin_callback)(struct whisper_context * ctx, void * user_data);

Would this work?

[taf2]

ah perfect - i'll dig into this and rework the ruby bindings to use that shortly.

[ggerganov]

Usage of the abort mechanism is demonstrated here:

whisper.cpp/examples/main/main.cpp

Lines 603 to 616 in b4ebdb6

	// example for abort mechanism
	// in this example, we do not abort the processing, but we could if the flag is set to true
	// the callback is called before every encoder run - if it returns false, the processing is aborted
	{
	static bool is_aborted = false; // NOTE: this should be atomic to avoid data race
	wparams.encoder_begin_callback = [](struct whisper_context * /*ctx*/, void * user_data) {
	bool is_aborted = *(bool*)user_data;
	return !is_aborted;
	};
	wparams.encoder_begin_callback_user_data = &is_aborted;
	}

[brauliobo]

am I missing something? It doesn't install here. I have whisper.cpp AUR package installed on the system

braulio @ bhavapower ➜  ruby-whisper.cpp git:(main) ✗  LANG=en gem install whispercpp
Building native extensions. This could take a while...
ERROR:  Error installing whispercpp:
        ERROR: Failed to build gem native extension.

    current directory: /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0/ext
/home/braulio/.rvm/rubies/ruby-3.1.2/bin/ruby -I /home/braulio/.rvm/rubies/ruby-3.1.2/lib/ruby/3.1.0 -r ./siteconf20240926-2567174-n68uhi.rb extconf.rb
cp: cannot stat './../../../whisper.cpp': No such file or directory
cp: cannot stat './../../../whisper.h': No such file or directory
cp: cannot stat './../../../ggml.h': No such file or directory
cp: cannot stat './../../../ggml.c': No such file or directory
cp: cannot stat './../../../ggml-impl.h': No such file or directory
cp: cannot stat './../../../ggml-alloc.h': No such file or directory
cp: cannot stat './../../../ggml-alloc.c': No such file or directory
cp: cannot stat './../../../ggml-backend-impl.h': No such file or directory
cp: cannot stat './../../../ggml-backend.h': No such file or directory
cp: cannot stat './../../../ggml-backend.c': No such file or directory
cp: cannot stat './../../../ggml-common.h': No such file or directory
cp: cannot stat './../../../ggml-quants.h': No such file or directory
cp: cannot stat './../../../ggml-quants.c': No such file or directory
cp: cannot stat './../../../examples/dr_wav.h': No such file or directory
creating Makefile

current directory: /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0/ext
make DESTDIR\= clean

current directory: /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0/ext
make DESTDIR\=
compiling ggml.c
ggml.c:4:10: fatal error: ggml-impl.h: No such file or directory
    4 | #include "ggml-impl.h"
      |          ^~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:246: ggml.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/braulio/.rvm/gems/ruby-3.1.2/gems/whispercpp-1.3.0 for inspection.
Results logged to /home/braulio/.rvm/gems/ruby-3.1.2/extensions/x86_64-linux/3.1.0/whispercpp-1.3.0/gem_make.out

[oriolgual]

I am also getting this error, how did you fix it?

[brauliobo]

just pushed a new gem at https://rubygems.org/gems/whisper.cpp

[oriolgual]

Hi! Can this be installed without CUDA? I'm getting this error:

Cloning whisper.cpp repository...
Cloning into 'whisper.cpp'...
Building libwhispercpp.so with GGML_CUDA=1...
I ccache not found. Consider installing it for faster compilation.
expr: syntax error
I whisper.cpp build info:
I UNAME_S:   Darwin
I UNAME_P:   arm
I UNAME_M:   arm64
I CFLAGS:    -Iggml/include -Iggml/src -Iinclude -Isrc -Iexamples -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -DNDEBUG -DGGML_USE_ACCELERATE -DGGML_USE_BLAS -DACCELERATE_NEW_LAPACK -DACCELERATE_LAPACK_ILP64 -DGGML_USE_CUDA -I/usr/local/cuda/include -I/usr/local/cuda/targets/arm64-linux/include  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -pthread -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion
I CXXFLAGS:  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi -Iggml/include -Iggml/src -Iinclude -Isrc -Iexamples -D_XOPEN_SOURCE=600 -D_DARWIN_C_SOURCE -DNDEBUG -DGGML_USE_ACCELERATE -DGGML_USE_BLAS -DACCELERATE_NEW_LAPACK -DACCELERATE_LAPACK_ILP64 -DGGML_USE_CUDA -I/usr/local/cuda/include -I/usr/local/cuda/targets/arm64-linux/include
I NVCCFLAGS: -std=c++11 -O3 -use_fast_math --forward-unknown-to-host-compiler -arch=native -DGGML_CUDA_DMMV_X=32 -DGGML_CUDA_MMV_Y=1 -DK_QUANTS_PER_ITERATION=2 -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128
I LDFLAGS:   -framework Accelerate -lcuda -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/usr/lib64 -L/usr/local/cuda/targets/arm64-linux/lib -L/usr/local/cuda/lib64/stubs -L/usr/lib/wsl/lib
I CC:        Apple clang version 16.0.0 (clang-1600.0.26.3)
I CXX:       Apple clang version 16.0.0 (clang-1600.0.26.3)
/bin/sh: nvcc: command not found
I NVCC:
/bin/sh: nvcc: command not found
grep: invalid option -- P
usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]]
	[-e pattern] [-f file] [--binary-files=value] [--color=when]
	[--context[=num]] [--directories=action] [--label] [--line-buffered]
	[--null] [pattern] [file ...]
Makefile:855: *** I ERROR: For CUDA versions < 11.7 a target CUDA architecture must be explicitly provided via environment variable CUDA_DOCKER_ARCH, e.g. by running "export CUDA_DOCKER_ARCH=compute_XX" on Unix-like systems, where XX is the minimum compute capability that the code needs to run on. A list with compute capabilities can be found here: https://developer.nvidia.com/cuda-gpus .  Stop.
Failed to build libwhisper.a and libggml.a

extconf failed, exit code 1

[brauliobo]

@oriolgual unfortunately not, currently GGML_CUDA=1 is set on the extconf.rb. I'm not sure it is a good idea to allow the user to set this environment variable on the build.

[oriolgual]

And why is CUDA required?

[brauliobo]

it is just my use case here. would be ideal if whisper.cpp could be compiled with everything and then cuda is selected on runtime

[KitaitiMakoto]

[ggerganov]

ggml-backend.c is old and should be removed. It was changed to C++ ggml-backend.cpp

[KitaitiMakoto]

@ggerganov
Oops. I removed my comment because it should be on Issue. I filed my removed comment as an issue: #2483
I'm sorry for removing it.

BTW, thanks for your quick response.