Fixed entire application. Should work as intended now. #612

leaskeg · 2024-07-06T23:59:22Z

Summary

This pull request addresses an issues where the repo is not working anymore. Tested locally and it works for me as it did before.

Changes

Updated the version number
Updated download_driver to support the new version of undetected_chromedriver
Converts major_version to an int so it doesnt cause an issue
ChromeOptions updated
Removed the loggingPrefs capability from the WebDriver initialization since it was causing an issue with the new version.
The loggingPrefs capability was causing an invalid argument: unrecognized capability error. Removing this capability resolves the issue and allows multiple workers to initialize correctly.
Updated Requirements.txt

Testing

Tested locally by running the script with multiple workers to ensure that the error no longer occurs and that the script functions as expected.

I think thats pretty much it. Works like before - tested with proxies from webshare

Please review and merge this to fix the application

… loggingPrefs and improved error handling. Working as of 07-07-2024 by Leaske

Should be its original values

should be its orignal value, this was used for testing

Removed proxies, was used for testing

leaskeg

@MShawon

AminDhouib · 2024-07-07T00:12:17Z

Appreciate the effort you put there, @leaskeg, though it seems like there are still some issues when I run it. It attempts to get the chrome driver automatically but fails to do so. I attached a screenshot of the error, including the written text.

PS C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer> py -3.10 .\youtube_viewer.py

Yb  dP  dP"Yb  88   88 888888 88   88 88""Yb 888888
 YbdP  dP   Yb 88   88   88   88   88 88__dP 88__
  8P   Yb   dP Y8   8P   88   Y8   8P 88""Yb 88""
 dP     YbodP  `YbodP'   88   `YbodP' 88oodP 888888

                        Yb    dP 88 888888 Yb        dP 888888 88""Yb
                         Yb  dP  88 88__    Yb  db  dP  88__   88__dP
                          YbdP   88 88""     YbdPYbdP   88""   88"Yb
                           YP    88 888888    YP  YP    888888 88  Yb


           [ GitHub : https://github.com/MShawon/YouTube-Viewer ]


+-------------------------- Version: 1.8.0 --------------------------+

Getting Chrome Driver...
Traceback (most recent call last):
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtube_viewer.py", line 1003, in <module>
    osname, exe_name = download_driver(patched_drivers=patched_drivers)
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtubeviewer\download_driver.py", line 112, in download_driver
    uc.install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 262, in install
    ChromeDriverManager(executable_path, target_version, *args, **kwargs).install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 192, in install
    self.fetch_chromedriver()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 221, in fetch_chromedriver
    ver = self.get_release_version_number().vstring
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 211, in get_release_version_number
    return LooseVersion(urlopen(self.__class__.DL_BASE + path).read().decode())
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 525, in open
    response = meth(req, response)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 634, in http_response
    response = self.parent.error(
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 563, in error
    return self._call_chain(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

leaskeg · 2024-07-07T00:17:30Z

Appreciate the effort you put there, @leaskeg, though it seems like there are still some issues when I run it. It attempts to get the chrome driver automatically but fails to do so. I attached a screenshot of the error, including the written text.

PS C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer> py -3.10 .\youtube_viewer.py

Yb  dP  dP"Yb  88   88 888888 88   88 88""Yb 888888
 YbdP  dP   Yb 88   88   88   88   88 88__dP 88__
  8P   Yb   dP Y8   8P   88   Y8   8P 88""Yb 88""
 dP     YbodP  `YbodP'   88   `YbodP' 88oodP 888888

                        Yb    dP 88 888888 Yb        dP 888888 88""Yb
                         Yb  dP  88 88__    Yb  db  dP  88__   88__dP
                          YbdP   88 88""     YbdPYbdP   88""   88"Yb
                           YP    88 888888    YP  YP    888888 88  Yb


           [ GitHub : https://github.com/MShawon/YouTube-Viewer ]


+-------------------------- Version: 1.8.0 --------------------------+

Getting Chrome Driver...
Traceback (most recent call last):
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtube_viewer.py", line 1003, in <module>
    osname, exe_name = download_driver(patched_drivers=patched_drivers)
  File "C:\Users\amind\OneDrive\Desktop\Projects\YouTube-Viewer\youtubeviewer\download_driver.py", line 112, in download_driver
    uc.install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 262, in install
    ChromeDriverManager(executable_path, target_version, *args, **kwargs).install()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 192, in install
    self.fetch_chromedriver()
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 221, in fetch_chromedriver
    ver = self.get_release_version_number().vstring
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\_compat.py", line 211, in get_release_version_number
    return LooseVersion(urlopen(self.__class__.DL_BASE + path).read().decode())
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 525, in open
    response = meth(req, response)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 634, in http_response
    response = self.parent.error(
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 563, in error
    return self._call_chain(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 496, in _call_chain
    result = func(*args)
  File "C:\Users\amind\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Could you attempt to download the chromedriver manually and add it to the folder? Much like below? I will attempt to work on the automatic version. But for now, this should be working. Fingers crossed.

…ouTube-Viewer into fix-entire-application

leaskeg · 2024-07-07T10:42:39Z

Hope that sorts the issues with getting chromedriver. Remember to follow MShawon's installation guide:

AminDhouib · 2024-07-07T16:33:39Z

I manually inserted a ChromeDriver and gave it a fixed path using driver_executable_path.
However, the script should support adding ChromeDrivers automatically.

For the time being, I installed ChromeDriver and called it chromedriver1.exe. Source for download:

Replaced the following lines at line 113:

    driver = uc.Chrome(
        driver_executable_path="./chromedriver1.exe",
        options=options,
        major_version=major_version,
    )

It works so far. But yeah, it should automatically download Chrome drivers seamlessly. I have other scripts currently using undetected Selenium that always download the latest Chrome driver correctly, so the issue lies with how it is implemented now.

Note I am using Windows.

leaskeg · 2024-07-07T16:50:20Z

I manually inserted a ChromeDriver and gave it a fixed path using driver_executable_path. However, the script should support adding ChromeDrivers automatically.

For the time being, I installed ChromeDriver and called it chromedriver1.exe. Source for download:

https://storage.googleapis.com/chrome-for-testing-public/126.0.6478.126/win64/chromedriver-win64.zip

https://googlechromelabs.github.io/chrome-for-testing/

Replaced the following lines at line 113:
    driver = uc.Chrome(
        driver_executable_path="./chromedriver1.exe",
        options=options,
        major_version=major_version,
    )
It works so far. But yeah, it should automatically download Chrome drivers seamlessly. I have other scripts currently using undetected Selenium that always download the latest Chrome driver correctly, so the issue lies with how it is implemented now.

Note I am using Windows.

I will attempt to make the automatic download work sometime next week :) Glad it works manually tho.

AminDhouib · 2024-07-07T17:57:47Z

Thanks, @leaskeg, for your efforts! And I appreciate what you have done to bring the project back to life.

I am curious about your current fixes. Are the views still active on your analytics?

leaskeg · 2024-07-07T19:59:26Z

Thanks, @leaskeg, for your efforts! And I appreciate what you have done to bring the project back to life.

I am curious about your current fixes. Are the views still active on your analytics?

They are actually! Tried with 100 views from webshares proxies. They counted.

knofte · 2024-07-07T20:32:03Z

leaskeg · 2024-07-07T22:21:27Z

@leaskeg And if I get " | HTTP | Random | Lat : 13.8196 | Lon : 100.04427 | TZ: Asia/Bangkok | Detected? : True " the detected = true, I guess it's because of proxies not because of the app? (linux)

I will get back to you on that. Its possible that it detects the browser as being automated which is not ideal, because youtube would be able to detect the views as bots. But that would indicate a problem with the undetected-chromedriver im using. I will look it over!

leaskeg · 2024-07-07T22:48:42Z

I have fixed the detection issue. I will attempt to fix the automatic chromedriver mentioned above before releasing. And thank you for all the feedbacks. It really helps, you can only test so much locally 👍

…y download

leaskeg · 2024-07-08T12:40:43Z

This version should hopefully fix the issues people are mentioning here in the comments. I tested locally and it downloads the chromedriver automatically, and its also undetected. Test it out and let me know

knofte · 2024-07-08T13:32:46Z

It works, ( undetected False) however it started triggering a few other issues :)

inside chrome window:
"Check if there is a typo in user-agent=mozilla"

I'll continue test though :)

leaskeg · 2024-07-08T17:54:20Z

This is the analytics while i was simply testing a few things. Seems to work as intended. Will look into the errors above.

zaid404 · 2024-08-18T06:36:48Z

Hi @leaskeg,

Thanks!
some log..
!./chromedriver --version
!google-chrome-stable --version
!./patched_drivers/chromedriver_0 --version
!lsb_release -a

ChromeDriver 126.0.6478.126 (d36ace6122e0a59570e258d82441395206d60e1c-refs/branch-heads/6478@{#1591})
Google Chrome for Testing 126.0.6478.126
ChromeDriver 126.0.6478.126 (d36ace6122e0a59570e258d82441395206d60e1c-refs/branch-heads/6478@{#1591})
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.3 LTS
Release: 22.04
Codename: jammy

leaskeg and others added 5 commits July 7, 2024 01:51

Fixed chromedriver not working with the new version. Fixed issue with…

dce5940

… loggingPrefs and improved error handling. Working as of 07-07-2024 by Leaske

Update search.txt

081729b

Should be its original values

Update search.txt

67caeb7

Update urls.txt

b5bdf1c

should be its orignal value, this was used for testing

Update proxy.txt

2ee4a8d

Removed proxies, was used for testing

leaskeg commented Jul 7, 2024

View reviewed changes

leaskeg changed the title ~~Fixed chromedriver not working with the new version. Fixed issue with…~~ Fixed entire application. Should work as intended now. Jul 7, 2024

leaskeg added 4 commits July 7, 2024 12:32

updated requirements.txt - completely forgot -.-

5245907

Merge branch 'fix-entire-application' of https://github.com/leaskeg/Y…

0e16c8e

…ouTube-Viewer into fix-entire-application

Updated

9bf61fb

Updated Requirements

c0521ba

Undetected chromedriver now works correctly, should also automaticall…

77f51e0

…y download

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed entire application. Should work as intended now. #612

Fixed entire application. Should work as intended now. #612

leaskeg commented Jul 6, 2024 •

edited

Loading

leaskeg left a comment •

edited

Loading

AminDhouib commented Jul 7, 2024

leaskeg commented Jul 7, 2024

leaskeg commented Jul 7, 2024

AminDhouib commented Jul 7, 2024 •

edited

Loading

leaskeg commented Jul 7, 2024

AminDhouib commented Jul 7, 2024

leaskeg commented Jul 7, 2024

knofte commented Jul 7, 2024

leaskeg commented Jul 7, 2024

leaskeg commented Jul 7, 2024 •

edited

Loading

leaskeg commented Jul 8, 2024

knofte commented Jul 8, 2024 •

edited

Loading

leaskeg commented Jul 8, 2024 •

edited

Loading

zaid404 commented Aug 18, 2024 •

edited

Loading

Fixed entire application. Should work as intended now. #612

Are you sure you want to change the base?

Fixed entire application. Should work as intended now. #612

Conversation

leaskeg commented Jul 6, 2024 • edited Loading

Summary

Changes

Testing

leaskeg left a comment • edited Loading

Choose a reason for hiding this comment

AminDhouib commented Jul 7, 2024

leaskeg commented Jul 7, 2024

leaskeg commented Jul 7, 2024

AminDhouib commented Jul 7, 2024 • edited Loading

leaskeg commented Jul 7, 2024

AminDhouib commented Jul 7, 2024

leaskeg commented Jul 7, 2024

knofte commented Jul 7, 2024

leaskeg commented Jul 7, 2024

leaskeg commented Jul 7, 2024 • edited Loading

leaskeg commented Jul 8, 2024

knofte commented Jul 8, 2024 • edited Loading

leaskeg commented Jul 8, 2024 • edited Loading

zaid404 commented Aug 18, 2024 • edited Loading

leaskeg commented Jul 6, 2024 •

edited

Loading

leaskeg left a comment •

edited

Loading

AminDhouib commented Jul 7, 2024 •

edited

Loading

leaskeg commented Jul 7, 2024 •

edited

Loading

knofte commented Jul 8, 2024 •

edited

Loading

leaskeg commented Jul 8, 2024 •

edited

Loading

zaid404 commented Aug 18, 2024 •

edited

Loading