atuin not saving new commands into local database

[ben-alkov]

I installed atuin per the README on 2022-04-28, in what I presume is local-only mode (i.e. I didn't set up any kind of sync).
I then imported my bash history, which worked great (very fast as well).
Today (2022-05-02), I noticed that recent commands seem to be missing.
~/.local/share/atuin/ looks like...

dir. .local/share/atuin/
total 6.2M
-rw-r--r--. 1 xxxx xxxx 1.9M Apr 28 17:20 history.db
-rw-r--r--. 1 xxxx xxxx  32K May  2 16:09 history.db-shm
-rw-r--r--. 1 xxxx xxxx 4.3M Apr 28 17:21 history.db-wal
# 'xxxx' is my redacted local username

...so indeed the DB isn't being updated. Config is straight defaults.
(Note that I rebooted a couple of hours ago, so ALL shells have been restarted :D)
Am I missing some configuration magic?

Footnote: The fact that history.db-shm has a current timestamp suggests to me that atuin is doing its job in writing new entries, but given that I rebooted, I should think that the transactions would've been written back...

[conradludgate]

Hmm, very perplexing. Just to ensure that the setup is valid, could you check for the existence of the ATUIN_SESSION environment variable?

[ellie]

Oh interesting 🤔 The WAL file should be getting periodically written to the main database, I wonder why it's not happening here

Could you try opening the sqlite file with

sqlite3 ~/.local/share/atuin/history.db

and then run

pragma wal_checkpoint(truncate);

and share the output please :)

This should force a checkpoint, and then truncate the wal file. See: https://www.sqlite.org/pragma.html#pragma_wal_checkpoint

[ben-alkov]

ATUIN_SESSION environment variable

Exists

pragma wal_checkpoint(truncate);

0|0|0

[ben-alkov]

Note that I have since disabled atuin, started a new shell, blown away the old db, re-ran atuin import bash, and re-enabled atuin. "Live" history append seemed to work briefly, but stopped again at some point.

[peterjanes]

I'm using sync and seeing the same issue. As with Ben's report, ATUIN_SESSION exists and the checkpoint returned 0|0|0 (and updated the timestamp on history.db and emptied history.db-wal). I also rebooted this morning.

Another data point of interest: I haven't overridden the default sync, but last_sync_time is currently >1h ago, coincidentally (?) just before I rebooted.

$ ls -la ~/.local/share/atuin/
total 708
drwxr-xr-x.  2 xxxx xxxx    116 May  3 11:36 .
drwx--x--x. 48 xxxx xxxx   4096 May  2 15:12 ..
-rw-r--r--.  1 xxxx xxxx 675840 May  3 11:18 history.db
-rw-r--r--.  1 xxxx xxxx  32768 May  3 11:36 history.db-shm
-rw-r--r--.  1 xxxx xxxx      0 May  3 11:36 history.db-wal
-rw-r--r--.  1 xxxx xxxx     48 May  2 14:36 key
-rw-r--r--.  1 xxxx xxxx     35 May  3 10:22 last_sync_time
-rw-r--r--.  1 xxxx xxxx     32 May  2 14:36 session
$ cat ~/.local/share/atuin/last_sync_time 
2022-05-03T14:22:51.151997318+00:00

[uthark]

I have the same issues, reproducible on all 3 different hosts (macOS 11 and macOS 12).
I use zsh with oh-my-zsh.

[ellie]

So it sounds like this is happening on both Bash and Zsh?

1. does the output of atuin history list show your history?
2. does sqlite3 ~/.local/share/atuin/history.db "select * from history order by timestamp asc;" show your history? The last item should have a couple of -1s after the timestamp.

@peterjanes, if you run atuin sync, does that complete successfully + is the last_sync_time updated?

I'm wondering if there's something funky going on with the shell hooks

[peterjanes]

1. atuin history list doesn't show my latest commands. The most recent has a timestamp that's just before the last_sync_time.
2. The last entry in the sql query is the same as in atuin history list. I see some -1 entries way back (looks like one of my imported histories) but not at the end.
3. atuin sync updated the last_sync_time but didn't change the timestamp on history.db

[ellie]

Hmmm interesting.

could anyone with this issue try running

export RUST_LOG=debug

and then see what's logged? Might shed light onto things. We do also need to sort out the logging experience 😅

Otherwise - what version of Atuin are people running, and which shell?

I'd basically like to determine if this is atuin not being called properly in the first place, or if it IS being called, and not inserting for some reason. A reboot would have triggered a new login, and hence a new load of .zprofile or .bash_profile, which could be messing with things here

[peterjanes]

$ echo $BASH_VERSION 
5.1.8(1)-release

$ atuin --version
atuin 0.9.1

I don't see any log messages or files being created with the RUST_LOG variable set. However, I have seen this message occasionally (without the variable) and I'm pretty sure it's new since atuin:

/home/xxx/.bashrc: line 96: bind: warning: line editing not enabled

Curiously, my ~/.bashrc only has 83 lines.

[peterjanes]

Aha, a data point here: I'm using bash-git-prompt and if I disable it in my .bash_profile atuin starts to update its records again. (It's executed after atuin.)

# Git prompt
GIT_PROMPT_ONLY_IN_REPO=1
source ~/.bash-git-prompt/gitprompt.sh

[conradludgate]

I don't see any log messages or files being created with the RUST_LOG variable set

This would imply that atuin is not running at all (the hooks would not be enabled)

if I disable it in my .bash_profile atuin starts to update its records again

Ah ok! Now we are somewhere.

Atuin for bash should be additive to your shell hooks. So you should be able to enable it after bash-git-prompt. I'm guessing they have a bad init that overrides the hooks

[peterjanes]

Can't speak for the zsh-ers but moving the init for bash-git-prompt worked here.

[jovandeginste]

I use (and combined) with starship.rs; I have the same issues.

[ellie]

I'm using starship.rs as well, on zsh, with no issues

Bash doesn't actually offer the hooks we need out of the box - so we use this - https://github.com/rcaloras/bash-preexec

@jovandeginste what shell is that?

[jovandeginste]

I'm using starship.rs as well, on zsh, with no issues

Bash doesn't actually offer the hooks we need out of the box - so we use this - https://github.com/rcaloras/bash-preexec

@jovandeginste what shell is that?

bash

[jovandeginste]

I tried to clean a bit up (I used fzf bindings before). I do have https://github.com/rcaloras/bash-preexec for starship command support

[jovandeginste]

I think that did the trick for me... disabled the bash pre-exec (I got it from here)

[jovandeginste]

... and now I upgraded that script (from V0.4.1 to V0.5.0) and it seems to work...!

[ben-alkov]

I have a fairly complex (mostly) homemade prompt which uses (among other things) git-radar, oc (for OpenShift/k8s), a local fn for formatting, and of course PROMPT_COMMAND.
I read the usage notes for bash-preexec.sh - they're rather contradictory:

"Source this file at the end of your bash profile so as not to interfere with...PROMPT_COMMAND"

but then also

"This module requires...Bash features which you must not otherwise be using...the "PROMPT_COMMAND" variable. If you override...these after bash-preexec has been installed it will most likely break"

TL;DR - I probably have some PROMPT_COMMAND debug/engineering to do

[ellie]

I'm going to close this one for now, it doesn't look like there's much we can do

If you're facing this problem, please ensure that your PROMPT and bash-preexec are not conflicting with each other! You may need to change the order of some commands in your bashrc.

[sftblw]

Maybe not related but for information, I had similar symptom and the reason for me was triple initialization by running the installer script three times over time. The first one had manual edit for disabling up arrow.

~/.bashrc

[[ -f ~/.bash-preexec.sh ]] && source ~/.bash-preexec.sh
eval "$(atuin init bash --disable-up-arrow)"

[[ -f ~/.bash-preexec.sh ]] && source ~/.bash-preexec.sh
eval "$(atuin init bash)"

[[ -f ~/.bash-preexec.sh ]] && source ~/.bash-preexec.sh
eval "$(atuin init bash)"

[cameronj86]

@sftblw's fix worked for me as well.

For me, I had manually pasted atuin init bash into .bashrc to override the system not accepting adjustments to the key bindings and inadvertently commented out the existing part of .bashrc:

[[ -f ~/.bash-preexec.sh ]] && source ~/.bash-preexec.sh

The errors I was receiving prior to this were:

1. Infrequent updates that were dumped all at once (ie: 40 commands all have the same timestamp
2. Commands didn't instantly show up in atuin until if/when they were sync'd

Unfortunately, now it syncs but the key_binding has gone back to opening atuin via up-arrow

[czue]

I just ran into this setting up a new machine. In my case the issue was not having bash-preexec installed, and the solution was to run:

curl https://raw.githubusercontent.com/rcaloras/bash-preexec/master/bash-preexec.sh -o ~/.bash-preexec.sh

[philippotto]

For me the solution was to remove the per-directory-history plugin I had installed for zsh. The existence of that plugin caused only empty commands to be saved into the atuin database.

[jimwins]

I ran into this problem when an old version (0.4.1) of bash-preexec had been loaded by another package (WezTerm) and then the newer version (0.5.0) was loaded, but the that version implements the double-include guard in a way that doesn't actually work with old versions if they are loaded first.

WezTerm has updated their bundled version of bash-preexec, and this pull request against bash-preexec would at least avoid the double-loading even though it would leave the older version in place.

[cohml]

Brancd new Atuin user, also experiencing this problem. However I use Zsh, so much of the later discussion (e.g., .bash-preexec.sh) is irrelevant for me, I think.

I followed the recommended instructions to install Atuin:

bash -c "<(curl https://raw.githubusercontent.com/atuinsh/atuin/main/install.sh)"

Then, I registered and ran atuin login, followed by running

atuin import auto
atuin sync --force

I now see Atuin, loaded with all of my old history, and the keybindings work as desired.

However, when I execute new commands, they don't show up in Atuin. I do see them in history | tail, and when I disable Atuin's up arrow keybinding, I can scroll up to access those new commands. But Atuin seems completely oblivious to their existence. Basically all I see are the old commands that I synced in.

I have attempted some light debugging, consisting mostly of deactivating plugins to avoid muddying the waters. But the problem persists.

Can @ellie or any fellow Zsh users assist? Absolutely beautiful project btw :)

[cohml]

Following up on my earlier comment...

So I think I figured it out. Pre-Atuin, my .zshrc had setopt SHARE_HISTORY. When I remove that one line, new commands start showing up in Atuin and persisting across sessions.

Not sure if this is a known bug or something I was just doing wrong, but anyway, I seem to have resolved my issues for now!

Edit: Since Atuin is a radical new way of tracking history, I'd be curious to know the extent to which zsh's typical history options still have an impact. For example, HIST_FIND_NO_DUPS, HIST_IGNORE_ALL_DUPS, HIST_REDUCE_BLANKS, ... Does Atuin utilize those at all?

[ellie]

Pre-Atuin, my .zshrc had setopt SHARE_HISTORY. When I remove that one line, new commands start showing up in Atuin and persisting across sessions.

Oh interesting! Definitely a bug. Would you be able to open a new issue for this specific problem please? I'm going to lock this issue now, as this is getting quite difficult to follow

I'd be curious to know the extent to which zsh's typical history options still have an impact. For example, HIST_FIND_NO_DUPS, HIST_IGNORE_ALL_DUPS, HIST_REDUCE_BLANKS, ... Does Atuin utilize those at all?

It shouldn't impact us at all. Atuin uses shell hooks to record history, and doesn't interact with the actual shell history mechanism