Will multiple sequences be supported for LLamaCpp backend?

[alfie-nsugh]

With something like this

def with_acceleration(lm, n_reps) -> float:
    for i in range(n_reps):
        lm += prefix + gen(name='story_{i}', max_tokens=4) + " "
    return lm

start = time.time()
llm_gpt2_large_gpu + with_acceleration(10)
print(f"With guidance acceleration and token healing:", time.time() - start)

torch.cuda.empty_cache()

the different generations can be seen as different sequences. With something like LLamaCpp, you can do parallel decoding by changing
batch.n_seq_id[j] = 1.

Will optimizations like this be supported at some point?