Query captions/subtitles [e.g. video transcripts] from xklb-metadata.db #244

deldesir · 2024-08-31T03:55:14Z

🚀 Pull Request Overview:
This is an alternative PR to #152 designed for A/B testing. It allows searching querying xklb-metadata.db. The ~~video titles~~ ids returned as results are used to enhance Calibre-Web's simple search.

cf. https://github.com/chapmanjacobd/library/blob/main/xklb/mediadb/search.py

📋 Checklist:

Tested the changes on Ubuntu 24.04 with https://www.youtube.com/watch?v=V8Bn0UPpGCw using xklb v2.9.036

root@box:~# lb search /library/calibre-web/xklb-metadata.db phytoplankton --to-json
8 captions
Is this the most valuable thing in the ocean? - David Biello - /library/calibre-web/TED-Ed/Is this the most valuable thing in the ocean_ - David Biello (9)/Is this the most valuable thing in the oce - TED-Ed.mp4
    1:21 we need to look at some of the ocean’s most ubiquitous organisms:
         phytoplankton.
    1:41 phytoplankton are everywhere.. and phytoplankton blooms are large
         enough to be seen from space.. These phytoplankton then become
         food for countless microscopic grazers,
    2:11 But when phytoplankton die, their bodies can sink far below the
         surface,
    2:27 whales consume huge amounts of these phytoplankton predators.
    3:16 Phytoplankton’s massive photosynthetic blooms. phytoplankton
         helps sequester huge amounts of carbon—. According to one report,
         phytoplankton are estimated to capture four times. every 1%
         increase in phytoplankton population
    4:02 researchers are working to increase phytoplankton populations
    4:17 than supporting the ocean’s natural phytoplankton farmers.

📌 Testing scenario:

Download a single video or playlist
Use the search field to search for some word or phrase present in the subtitles. You might need to watch a video to get some terms ;-)

Related to #152

cc @EMG70

Function to search through subtitles in xklb-metadata.db. The video titles returned as results are used to enhance Calibre-Web's simple search.

holta · 2024-08-31T06:00:08Z

PR Support searching videos [subtitles, captions, transcripts?] #152
Search videos by subtitles/captions/transcripts [AND video title AND video description AND video author AND video publisher?] #197

holta · 2024-08-31T22:45:31Z

Should this PR be tested more broadly while comparing it to PR #152 now, as suggested at... ?

Support searching videos [subtitles, captions, transcripts?] #152 (comment)

(Or is this PR still a draft for good reason?)

deldesir · 2024-08-31T23:13:49Z

It's still a draft. Trying to emulate xklb's internals here but my tests aren't conclusive yet due to a misunderstanding of how fts works. I am refactoring this completely to mirror (steal) xklb's code, then will remove the features we don't want for simplicity sake. Worth it as I am learning a lot by doing this.

deldesir · 2024-09-02T06:56:51Z

Ready for testing.

holta · 2024-09-02T12:11:00Z

Ready for testing.

Thanks!

Can you summarize the differences from PR #152 ?

This is an alternative PR to #152 designed for A/B testing. It allows searching querying xklb-metadata.db

Does this PR #244 search both metadata.db and xklb-metadata.db for completeness?

deldesir · 2024-09-02T13:38:03Z

Ready for testing.

Thanks!

Can you summarize the differences from PR #152 ?

This PR doesn't run any command in the background to get its results output to STDOUT, #152 runs a command.

This is an alternative PR to #152 designed for A/B testing. It allows searching querying xklb-metadata.db

Does this PR #244 search both metadata.db and xklb-metadata.db for completeness?

No, as explained in #197 , this PR only searches xklb-metadata.db. Completeness is assured by calibre-web internal search which merges its own results with this PR's ones.

holta · 2024-09-02T18:01:05Z

Can you summarize the differences from PR #152 ?

This PR doesn't run any command in the background to get its results output to STDOUT, #152 runs a command.

@deldesir explain in a bit more detail please, for everyone?

How/why is this approach (PR #244) likely preferred over PR #152 ?

holta · 2024-09-03T04:33:38Z

@EMG70 would you have time to test this PR?

(And compare it to your impressions of PR #152 !)

EMG70 · 2024-09-03T06:15:11Z

I will test later on Today.

deldesir · 2024-09-03T14:06:29Z

This PR code does not execute any subprocess calls or run any external commands, #152 does. This PR is focused on querying and processing data within xklb-metadata.db, manipulating dictionaries, and formatting time strings.

The code is designed to interact with captions and media information. It return titles of media where captions match a search term.

This PR's approach is preferred because all operations are handled in-memory within Python without invoking any subprocesses (#152 invokes a subprocess to run lb search <db> <term> --to-json)

holta · 2024-10-19T19:24:16Z

Huge thanks @EMG70, for pushing everybody forward!

deldesir · 2024-10-19T19:28:11Z

Thanks again @EMG70. It looks like PR #259 is not pulled properly. Can you double check with the command:
git pull https://github.com/deldesir/calibre-web deldesir-xklb-schema --no-rebase --no-edit

Also run systemctl restart calibre-web after.

EMG70 · 2024-10-19T19:32:35Z

After running PR#259
root@box:/usr/local/calibre-web-py3# git pull https://github.com/deldesir/calibre-web deldesir-xklb-schema --no-rebase --no-edit
error: Pulling is not possible because you have unmerged files.
hint: Fix them up in the work tree, and then use 'git add/rm '
hint: as appropriate to mark resolution and make a commit.
fatal: Exiting because of an unresolved conflict.
root@box:/usr/local/calibre-web-py3#

deldesir · 2024-10-19T19:44:56Z

Can you do git status so I can know which file contains conflicts? Feel free to post the output or screenshot here.

EMG70 · 2024-10-19T19:47:19Z

root@box:/usr/local/calibre-web-py3# git status
On branch master
Your branch is ahead of 'origin/master' by 44 commits.
(use "git push" to publish your local commits)

You have unmerged paths.
(fix conflicts and run "git commit")
(use "git merge --abort" to abort the merge)

Unmerged paths:
(use "git add ..." to mark resolution)
both modified: cps/tasks/download.py

Untracked files:
(use "git add ..." to include in what will be committed)
bin/
include/
lib/
lib64
pyvenv.cfg

no changes added to commit (use "git add" and/or "git commit -a")
root@box:/usr/local/calibre-web-py3#

deldesir · 2024-10-19T19:54:06Z

Thanks. I would like to see one more output or screenshot to pin the problem.

Open cps/tasks/download.py by running nano /usr/local/calibre-web-py3/cps/tasks/download.py.
You'll see sections marked with conflict markers.

Copy paste these sections. They look like:

<<<<<<< HEAD 
======= 
>>>>>>>

A screeshot is ok if copy pasting is not possible

EMG70 · 2024-10-19T19:59:21Z

 <<<<<<< HEAD
 57
 58                 # Database operations
 59                 with sqlite3.connect(XKLB_DB_FILE) as conn:
 60                     try:
 61                         # requested_file = conn.execute("SELECT path FROM media WHERE webpath = ? AND path NOT LIKE 'http%'", (self.media_url,)).fetchone()[0]
 62                         self.media_id, requested_file = conn.execute("SELECT id, path FROM media WHERE webpath = ? AND path NOT LIKE 'http%'", (self.media_url,)).fetchone()
 63
 64                         # Abort if there is not a path
 65                         if not requested_file:
 66                             log.info("No path found in the database")
 67                             error = conn.execute("SELECT error, webpath FROM media WHERE error IS NOT NULL").fetchone()
 68                             if error:
 69                                 log.error("[xklb] An error occurred while trying to download %s: %s", error[1], error[0])
 70                                 self.message = f"{error[1]} failed to download: {error[0]}"
 71                             else:
 72                                 log.error("%s failed to download: No path or error found in the database (likely the video failed due to unavailable fragments?)", self.media_url)
 73                                 self.message = f"{self.media_url_link} failed to download: No path or error found in the database (likely the video failed due to unavailable fragm>
 74                                 self.media_id = conn.execute("SELECT id FROM media WHERE webpath = ?", (self.media_url,)).fetchone()[0]
 75                                 conn.execute("DELETE FROM media WHERE webpath = ?", (self.media_url,))
 76                                 conn.execute("DELETE FROM captions WHERE media_id = ?", (self.media_id,))
 77                             return
 78                     except sqlite3.Error as db_error:
 79                         log.error("An error occurred while trying to connect to the database: %s", db_error)
 80                         self.message = f"{self.media_url_link} failed to download: {db_error}"
 81
 82                     self.message = self.message + "\n" + f"Almost done..."
 83                     response = requests.get(self.original_url, params={"requested_file": requested_file, "current_user_name": self.current_user_name, "shelf_id": self.shelf_id, "m>
 84                     if response.status_code == 200:
 85                         log.info("Successfully sent the requested file to %s", self.original_url)
 86                         file_downloaded = response.json()["file_downloaded"]
 87                         self.message = f"Successfully downloaded {self.media_url_link} to <br><br>{file_downloaded}"
 88                         new_video_path = response.json()["new_book_path"]
 89                         new_video_path = next((os.path.join(new_video_path, file) for file in os.listdir(new_video_path) if file.endswith((".webm", ".mp4"))), None)
 90                         # 2024-02-17: Dedup Design Evolving... https://github.com/iiab/calibre-web/pull/125
 91                         conn.execute("UPDATE media SET path = ? WHERE webpath = ?", (new_video_path, self.media_url))
 92                         conn.execute("UPDATE media SET webpath = ? WHERE path = ?", (f"{self.media_url}&timestamp={int(datetime.now().timestamp())}", new_video_path))
 93                         self.progress = 1.0
 94                     else:
 95                         log.error("Failed to send the requested file to %s", self.original_url)
 96                         self.message = f"{self.media_url_link} failed to download: {response.status_code} {response.reason}"
 97
 98                 conn.close()
 99
100 =======
101                 self._post_process_download()
102 >>>>>>> 95a0d554e50fef881f129b4ed2d929249dbffd17

deldesir · 2024-10-19T20:02:29Z

I think I understand what happened. I'll be back with a solution in a couple of minutes. Thanks for your patience.

deldesir · 2024-10-19T20:30:47Z

It shouldn't be you to do this, but I'll ask you to fix your install with the instructions below. I must tell you the problem was caused by PR #255 proposing unecessary changes for download.py, not because you did something wrong. I understand now the order matters when testing PRs so I did my part by adjusting PR #255 to prevent this issue for others.

Open the download.py if not already opened with nano /usr/local/calibre-web-py3/cps/tasks/download.py
Remove <<<<<<< HEAD (should be at line 56)
Remove lines 100, 101, 102
Save and Exit from nano editor:
- press Ctrl + X.
- you will be prompted to save, press Y to confirm.
- Press Enter to accept the current file name.
Stage the file by doing git add cps/tasks/download.py
Commit your changes by running git commit -m "Resolved conflict in download.py"
Pull PR Begin to establish 1:1 mapping between media_id (xklb-metadata.db) <-> Calibre-Web's book_id (metadata.db) #255 again by running git pull https://github.com/deldesir/calibre-web deldesir-patch-23 --no-rebase --no-edit
Pull PR Model xklb-metadata.db using SQLAlchemy #259 again by running git pull https://github.com/deldesir/calibre-web deldesir-xklb-schema --no-rebase --no-edit
Restart Calibre-Web by running systemctl restart calibre-web

holta · 2024-10-19T20:54:21Z

It shouldn't be you to do this

Indeed! This is a struggle we're currently sorting out, and let's all be patient getting this right! And in future:

Testing volunteers should NEVER get involved with merge conflicts. ✅
Test Plan / Test Instructions need to be FULLY SMOKE-TESTED in advance. ✅

EMG70 · 2024-10-19T21:08:02Z

I have followed your instructions .
SUDO IIAB-DIAGNOSTICS - https://paste.centos.org/view/ba1d5609
calibre-web still not reachable.

root@box:/usr/local/calibre-web-py3# systemctl restart calibre-web
root@box:/usr/local/calibre-web-py3# systemctl status calibre-web
× calibre-web.service - Calibre-Web
Loaded: loaded (/etc/systemd/system/calibre-web.service; enabled; preset: enabled)
Active: failed (Result: exit-code) since Sat 2024-10-19 22:07:10 BST; 6s ago
Duration: 1.330s
Process: 2247 ExecStart=/usr/local/calibre-web-py3/bin/python3 /usr/local/calibre-web-py3/cps.py -p /library/calibre-web/config/app.db (code=exited, status=1/FAILURE)
Main PID: 2247 (code=exited, status=1/FAILURE)
CPU: 1.328s

Oct 19 22:07:10 box python3[2247]: File "/usr/local/calibre-web-py3/cps/init.py", line 42, in
Oct 19 22:07:10 box python3[2247]: from . import ub, db
Oct 19 22:07:10 box python3[2247]: File "/usr/local/calibre-web-py3/cps/db.py", line 52, in
Oct 19 22:07:10 box python3[2247]: from .services.xb_utils import CaptionSearcher
Oct 19 22:07:10 box python3[2247]: File "/usr/local/calibre-web-py3/cps/services/xb_utils.py", line 9, in
Oct 19 22:07:10 box python3[2247]: from cps.glue_db import GlueDB, MediaBooksMapping
Oct 19 22:07:10 box python3[2247]: ModuleNotFoundError: No module named 'cps.glue_db'
Oct 19 22:07:10 box systemd[1]: calibre-web.service: Main process exited, code=exited, status=1/FAILURE
Oct 19 22:07:10 box systemd[1]: calibre-web.service: Failed with result 'exit-code'.
Oct 19 22:07:10 box systemd[1]: calibre-web.service: Consumed 1.328s CPU time.
root@box:/usr/local/calibre-web-py3#

deldesir · 2024-10-19T21:40:55Z

Ok. Let's call it a failed test for now. On my side, I will be following the instructions step by step to ensure we eat the same food👌. I'll provide some feedback once done to better assist you. Thanks for your hard testing and patience.

holta · 2024-10-19T21:56:09Z

following the instructions step by step to ensure we eat the same food👌

Perfect. Thank you.

deldesir · 2024-10-19T21:57:17Z

I did the test following the instructions step by step: https://paste.centos.org/view/dc30dfdf
@EMG70, please do the test again when possible.

You can either do it with a new VM or do it with the current one instead by doing

cd /usr/local/
mv calibre-web-py3 calibre-web-py3.OLD
cd /opt/iiab/iiab
./runrole --reinstall calibre-web

... and then try the pulls again:

cd /usr/local/calibre-web-py3
git pull https://github.com/deldesir/calibre-web deldesir-search-captions --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-22 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-23 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-24 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-25 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-27 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-xklb-schema --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-31 --no-rebase --no-edit
iiab-update -f

Above instructions worked for me, but it they fail for you, feel free to let me know:

EMG70 · 2024-10-19T22:11:32Z

Just a quick note.I have installed another fresh VM ,this time a 24.10.It worked OK as per above original instuctions.I know @holta had suggested not to use 24.10.
Could this issue be confined to 24.10 LTS?
SUDO IIB-DIAGNOSTICS https://paste.centos.org/view/bc3e5380

Download of a single video failed though.

EMG70 · 2024-10-19T22:12:32Z

I did the test following the instructions step by step: https://paste.centos.org/view/dc30dfdf @EMG70, please do the test again when possible.

You can either do it with a new VM or do it with the current one instead by doing

* ` cd /usr/local/`

* `mv calibre-web-py3 calibre-web-py3.OLD`

* `cd /opt/iiab/iiab`

* `./runrole --reinstall calibre-web`

... and then try the pulls again:

cd /usr/local/calibre-web-py3
git pull https://github.com/deldesir/calibre-web deldesir-search-captions --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-22 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-23 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-24 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-25 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-27 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-xklb-schema --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-31 --no-rebase --no-edit
iiab-update -f

Above instructions worked for me, but it they fail for you, feel free to let me know:

Am on it now.

deldesir · 2024-10-19T22:21:47Z

Just a quick note.I have installed another fresh VM ,this time a 24.10.It worked OK as per above original instuctions.I know @holta had suggested not to use 24.10. Could this issue be confined to 24.10 LTS? SUDO IIB-DIAGNOSTICS https://paste.centos.org/view/bc3e5380

Download of a single video failed though.

Glad the instructions worked for 24.10.
Yes, you're right for the download failure as per issue #267

holta · 2024-10-19T22:25:04Z

WARNING: Avoid Ubuntu 24.10 for now, as all video downloads appear to fail due to "Download to IIAB" button fails every time on Ubuntu 24.10 — "An error occurred during the subprocess execution: 'NoneType' object is not subscriptable" and later "failed to download: None" — perhaps b/c xklb marked your video as deleted because it failed to download in previous tries? #267 !

▶️ @EMG70 Please AVOID Ubuntu 24.10 as explained above (yesterday): #244 (comment)

▶️ Testing needs to begin on Ubuntu 24.04 "so we're all eating the same food."

EMG70 · 2024-10-19T22:26:44Z

Update after @deldesir's latest instructions:
sudo iiab-diagnostics - https://paste.centos.org/view/4f127ac4

calibre-web now working OK.

step 3. root@box:/usr/local/calibre-web-py3# ls /library/calibre-web/xklb-metadata.db
/library/calibre-web/xklb-metadata.db
root@box:/usr/local/calibre-web-py3#

Step 4. ✔️
step 5. ✔️

Step 6. Also verify if metadata are extended and the media.path is actually a path instead of a URL for the required video — meaning that the download.py refactoring was effective, and functionality is preserved.

Step 7.To further test, play a video and note the number in the URL for the video played from within Calibre-Web (e.g. for http://<IIAB_HOSTNAME_OR_IP>/books/read/31.mp4, the video ID is 31). Note also the title of the video, and note the media ID associated with it in iiab-glue.db.

Step 8.Find 2 videos with overlapping words in the title, but different captions

Step 9.Search for a term that's in the captions of one video but not the other. Only the video where the caption matches should return.

NB: I had downloaded more than 2 videos so results may not be as clear as expected,hope that gives an idea.

EMG70 · 2024-10-19T22:27:19Z

WARNING: Avoid Ubuntu 24.10 for now, as all video downloads appear to fail due to "Download to IIAB" button fails every time on Ubuntu 24.10 — "An error occurred during the subprocess execution: 'NoneType' object is not subscriptable" and later "failed to download: None" — perhaps b/c xklb marked your video as deleted because it failed to download in previous tries? #267 !

▶️ @EMG70 Please AVOID Ubuntu 24.10 as explained above (yesterday): #244 (comment)

▶️ Testing needs to begin on Ubuntu 24.04 "so we're all eating the same food."

Noted.

deldesir · 2024-10-19T22:50:16Z

Instructions updated to include running PIPX_BIN_DIR=/usr/local/bin pipx install datasette in Step 5.

Here are the complete instructions:

How to test

Install a brand new IIAB with Calibre-Web. We'd welcome tests on Ubuntu 24.04, Raspberry Pi OS or any other Debian-based distro running on old laptops.
- WARNING: Avoid Ubuntu 24.10 for now, as all video downloads appear to fail due to "Download to IIAB" button fails every time on Ubuntu 24.10 and 25.04 pre-releases — originally "An error occurred during the subprocess execution: 'NoneType' object is not subscriptable" — and now "failed to download: None" #267 !

Pull in the 8 PRs: (background: How do I test a pull request (PR) ?)

sudo -i
cd /usr/local/calibre-web-py3
git pull https://github.com/deldesir/calibre-web deldesir-search-captions --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-22 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-23 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-24 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-25 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-27 --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-xklb-schema --no-rebase --no-edit
git pull https://github.com/deldesir/calibre-web deldesir-patch-31 --no-rebase --no-edit
iiab-update -f

Just FYI this means you should be testing these 8 PRs, all together:

Before downloading a video, do ls /library/calibre-web/xklb-metadata.db in your Terminal to check for the db existence. If it exists, that means the SQLAlchemy successfully created it, attesting to the effectiveness of the database modeling.
Submit a YouTube video URL via the “Download to IIAB” button and monitor the “Tasks” page to check whether metadata fetching is reported.
On success, install datasette by running: PIPX_BIN_DIR=/usr/local/bin pipx install datasette
- Then review the extraction with datasette /library/calibre-web/xklb-metadata.db -h 0.0.0.0 -p 8080 by browsing to http://<IIAB_HOSTNAME_OR_IP>:8080
- If metadata are present for the required video, that means the metadata_extract.py refactoring was effective and functionality is preserved.
- Remember to type CTRL-c in the Linux Terminal where datasette is running, to end the datasette program as soon as you no longer need it, to free up port 8080 !
Also verify if metadata are extended and the media.path is actually a path instead of a URL for the required video — meaning that the download.py refactoring was effective, and functionality is preserved.
To further test, play a video and note the number in the URL for the video played from within Calibre-Web (e.g. for http://<IIAB_HOSTNAME_OR_IP>/books/read/31.mp4, the video ID is 31). Note also the title of the video, and note the media ID associated with it in iiab-glue.db.
- In another terminal window, run datasette /library/calibre-web/iiab-glue.db -h 0.0.0.0 -p 8080
- Open a new browser tab to go to http://<IIAB_HOSTNAME_OR_IP>:8080
- Remember to type CTRL-c in the Linux Terminal where datasette is running, to end the datasette program as soon as you no longer need it, to free up port 8080 !
- This media ID can be verified by inspecting xklb-metadata.db for its correctness. To do so with datasette, run datasette /library/calibre-web/xklb-metadata.db -h 0.0.0.0 -p 8080. Look for the ID row in media table and compare the title from the actual video with the record under media.title.
Find 2 videos with overlapping words in the title, but different captions.
Search for a term that's in the captions of one video but not the other. Only the video where the caption matches should return.
Wait 20mn, then download a video to check whether the erro "a transaction is already begun" is gone.
Do a search while a video is being downloaded to ensure that session management is working as expected.

EMG70 · 2024-10-19T23:02:09Z

I will continue steps 6-9 tomorrow.

holta · 2024-10-20T00:12:39Z

I just fixed the instructions for installing datasette in the Test Plan just above:

pipx install datasette ❌
PIPX_BIN_DIR=/usr/local/bin pipx install datasette ✅
PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install datasette if you want to follow our official instructions !

deldesir · 2024-10-22T14:00:36Z

@EMG70, did you get a 404 not found error when testing on 24.04. On my side, when I do the tests on Raspberry PI OS, I could not access Calibre-Web normally. I had to specify the Port 8083 to access it.

deldesir · 2024-10-22T18:45:52Z

Calibre-Web is now accessible on UNITTEST install on Raspberry Pi OS. All I had to do is:

cd /opt/iiab/iiab
./runrole network

EMG70 · 2024-10-22T19:02:48Z

@EMG70, did you get a 404 not found error when testing on 24.04. On my side, when I do the tests on Raspberry PI OS, I could not access Calibre-Web normally. I had to specify the Port 8083 to access it.

I have not yet tried latest PR tests on Raspberry PI.

holta · 2024-10-23T02:04:01Z

cd /opt/iiab/iiab
./runrole network

The official/recommended way of doing the above is to run the following as root: (the cd command is not necessary!)

iiab-network

(But possibly these 2 variables should be set in /etc/iiab/local_vars.yml first?)

network_install: True
network_enabled: True

deldesir · 2024-11-11T16:22:53Z

@EMG70 Test plan updated to check the fix for #271

deldesir added 9 commits March 28, 2024 10:57

Create lb_search.py

3d123a9

Function to search through subtitles in xklb-metadata.db. The video titles returned as results are used to enhance Calibre-Web's simple search.

Enhance results using xklb search

7ffe29d

Merge branch 'iiab:master' into deldesir-patch-26

b4475f7

Merge branch 'iiab:master' into deldesir-patch-26

42aafe8

Merge branch 'iiab:master' into deldesir-patch-26

9499ace

Add explanations

a5486db

Merge branch 'iiab:master' into deldesir-patch-26

ac60616

Get list of video titles

a4d0681

Query xklb-metadata directly

c4c16b6

deldesir self-assigned this Aug 31, 2024

deldesir marked this pull request as draft August 31, 2024 03:56

holta added the enhancement New feature or request label Aug 31, 2024

deldesir mentioned this pull request Aug 31, 2024

Support searching videos [subtitles, captions, transcripts?] #152

Open

1 task

deldesir added 2 commits September 2, 2024 02:33

Revert changes to align with iiab#152

b21001d

Query captions without subprocess

786b6a9

deldesir marked this pull request as ready for review September 2, 2024 06:55

deldesir requested a review from holta September 2, 2024 06:56

deldesir mentioned this pull request Sep 2, 2024

Search videos by subtitles/captions/transcripts [AND video title AND video description AND video author AND video publisher?] #197

Open

holta changed the title ~~Query captions/subtitles from xklb-metadata.db~~ Query captions/subtitles [a.k.a. transcripts] from xklb-metadata.db Sep 3, 2024

holta changed the title ~~Query captions/subtitles [a.k.a. transcripts] from xklb-metadata.db~~ Query captions/subtitles [e.g. video transcripts] from xklb-metadata.db Sep 3, 2024

deldesir mentioned this pull request Oct 28, 2024

Error downloading first video on an new install: no such table: main.media_fts #276

Closed

deldesir mentioned this pull request Nov 12, 2024

"Download to IIAB" button fails every time on Ubuntu 24.10 and 25.04 pre-releases — originally "An error occurred during the subprocess execution: 'NoneType' object is not subscriptable" — and now "failed to download: None" #267

Open

Query captions/subtitles [e.g. video transcripts] from xklb-metadata.db #244

Are you sure you want to change the base?

Query captions/subtitles [e.g. video transcripts] from xklb-metadata.db #244

Conversation

deldesir commented Aug 31, 2024 • edited Loading

holta commented Aug 31, 2024

holta commented Aug 31, 2024 • edited Loading

deldesir commented Aug 31, 2024

deldesir commented Sep 2, 2024

holta commented Sep 2, 2024 • edited by deldesir Loading

deldesir commented Sep 2, 2024 • edited Loading

holta commented Sep 2, 2024

holta commented Sep 3, 2024

EMG70 commented Sep 3, 2024

deldesir commented Sep 3, 2024 • edited Loading

holta commented Oct 19, 2024

deldesir commented Oct 19, 2024

EMG70 commented Oct 19, 2024 • edited Loading

deldesir commented Oct 19, 2024 • edited Loading

EMG70 commented Oct 19, 2024

deldesir commented Oct 19, 2024 • edited Loading

EMG70 commented Oct 19, 2024

deldesir commented Oct 19, 2024

deldesir commented Oct 19, 2024 • edited Loading

holta commented Oct 19, 2024

EMG70 commented Oct 19, 2024

deldesir commented Oct 19, 2024

holta commented Oct 19, 2024

deldesir commented Oct 19, 2024 • edited Loading

EMG70 commented Oct 19, 2024 • edited Loading

EMG70 commented Oct 19, 2024

deldesir commented Oct 19, 2024 • edited Loading

holta commented Oct 19, 2024 • edited Loading

EMG70 commented Oct 19, 2024 • edited Loading

EMG70 commented Oct 19, 2024

deldesir commented Oct 19, 2024 • edited Loading

EMG70 commented Oct 19, 2024

holta commented Oct 20, 2024

deldesir commented Oct 22, 2024

deldesir commented Oct 22, 2024

EMG70 commented Oct 22, 2024

holta commented Oct 23, 2024 • edited Loading

deldesir commented Nov 11, 2024

deldesir commented Aug 31, 2024 •

edited

Loading

holta commented Aug 31, 2024 •

edited

Loading

holta commented Sep 2, 2024 •

edited by deldesir

Loading

deldesir commented Sep 2, 2024 •

edited

Loading

deldesir commented Sep 3, 2024 •

edited

Loading

EMG70 commented Oct 19, 2024 •

edited

Loading

deldesir commented Oct 19, 2024 •

edited

Loading

deldesir commented Oct 19, 2024 •

edited

Loading

deldesir commented Oct 19, 2024 •

edited

Loading

deldesir commented Oct 19, 2024 •

edited

Loading

EMG70 commented Oct 19, 2024 •

edited

Loading

deldesir commented Oct 19, 2024 •

edited

Loading

holta commented Oct 19, 2024 •

edited

Loading

EMG70 commented Oct 19, 2024 •

edited

Loading

deldesir commented Oct 19, 2024 •

edited

Loading

holta commented Oct 23, 2024 •

edited

Loading