Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slim docker image #68

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

slim docker image #68

wants to merge 1 commit into from

Conversation

vogler
Copy link
Owner

@vogler vogler commented Feb 20, 2023

Current Docker image based on ubuntu:jammy is 1.12GB.

Tried out slim docker images with VNC (could be <400MB):

Alpine: installing deps and running firefox did not work - Playwright does not support Alpine: issue. Got /usr/bin/firefox to launch, but could not load profile.
Ubuntu: may work, but somehow could not install up-to-date nodejs to run npx playwright.

Other projects, but only supporting Chromium:

Alpine: installing deps and running firefox did not work
Ubuntu: may work, but could not install up-to-date nodejs
@GAS85
Copy link

GAS85 commented Jun 27, 2023

You can reduce 1.6 Mb by adding --no-cache-dir to pip install :D
I see that there is a lot of data been installed, FF is around 242 Mb, layer with VNC is already 644 Mb

P.S. you can see what was installed and how it is big with each layer with dive, simply do after installation:

dive ghcr.io/vogler/free-games-claimer

@vogler
Copy link
Owner Author

vogler commented Jun 27, 2023

Yea, I already checked all layers. The main problem is that Playwright does not support Alpine and Ubuntu with all its deps is quite heavy in comparison.
Could probably port some tricks (sep. build stages, upx, TigerVNC) from https://github.com/jlesage/docker-baseimage-gui/blob/master/Dockerfile, but seems like a lot of work.

@GAS85
Copy link

GAS85 commented Jun 28, 2023

There is not "tiny" image with Playwright at all... They are already around 674 Mb, so in our case we even more efficient. As this is most weight, I do not think that Alpine will give you benefit over Ubuntu here.

From the small points, we ca update command to not copy .git and .github folders to the container, e.g. by:

COPY *.* .

image

My personal Record is 965 vs 972 MB.

with this Dockerfile
FROM ubuntu:jammy as builder

RUN apt-get update \
    && apt-get install --no-install-recommends -y \
      curl \
      ca-certificates \
      dos2unix
WORKDIR /builder/fgc
COPY . .

# Shell scripts need Linux line endings. On Windows, git might be configured to check out dos/CRLF line endings, so we convert them for those people in case they want to build the image. They could also use --config core.autocrlf=input
# Remove uneeded folders, move entrypoint to the folder
RUN dos2unix *.sh && chmod +x *.sh \
    && rm -r .git* ; \
      mkdir -p ../usr/local/bin \
    && mv docker-entrypoint.sh ../usr/local/bin/

# FROM mcr.microsoft.com/playwright:v1.20.0
# Partially from https://github.com/microsoft/playwright/blob/main/utils/docker/Dockerfile.focal
FROM ubuntu:jammy

# Configuration variables are at the end!

# https://github.com/hadolint/hadolint/wiki/DL4006
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
ARG DEBIAN_FRONTEND=noninteractive

COPY --from=builder /builder/ /
WORKDIR /fgc

# Install up-to-date node & npm, deps for virtual screen & noVNC, firefox, pip for apprise.
RUN apt-get update \
    && apt-get install --no-install-recommends -y curl ca-certificates \
    && curl -fsSL https://deb.nodesource.com/setup_19.x | bash - \
    && apt-get install --no-install-recommends -y \
      nodejs \
      xvfb \
      x11vnc \
      tini \
      novnc websockify \
      python3-pip \
    # && npx playwright install-deps firefox \
    && apt-get install --no-install-recommends -y \
      libgtk-3-0 \
      libasound2 \
      libxcomposite1 \
      libpangocairo-1.0-0 \
      libpango-1.0-0 \
      libatk1.0-0 \
      libcairo-gobject2 \
      libcairo2 \
      libgdk-pixbuf-2.0-0 \
      libdbus-glib-1-2 \
      libxcursor1 \
    && apt-get autoclean -y \
    && apt-get autoremove -y \
    && rm -rf \
      /tmp/* \
      /usr/share/doc/* \
      /var/cache/* \
      /var/lib/apt/lists/* \
      /var/tmp/* \
    && find /var/log/ -type f -exec truncate -s 0 {} \; \
    && ln -s /usr/share/novnc/vnc_auto.html /usr/share/novnc/index.html

RUN pip --no-cache-dir install apprise

# Playwright installs patched firefox to ~/.cache/ms-playwright/firefox-*
# Requires some system deps to run (see install-deps above).
RUN npm install \
    && npm cache clean -f

# Old: PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD + install firefox (had to be done after `npm install` to get the correct version). Now: playwright-firefox as npm dep and `npm install` will only install that.
# RUN npx playwright install firefox

#COPY . .

# Shell scripts need Linux line endings. On Windows, git might be configured to check out dos/CRLF line endings, so we convert them for those people in case they want to build the image. They could also use --config core.autocrlf=input
#RUN dos2unix *.sh && chmod +x *.sh
#COPY docker-entrypoint.sh /usr/local/bin/

# Configure VNC via environment variables:
ENV VNC_PORT 5900
ENV NOVNC_PORT 6080
EXPOSE 5900
EXPOSE 6080

# Configure Xvfb via environment variables:
ENV WIDTH 1280
ENV HEIGHT 1280
ENV DEPTH 24

# Show browser instead of running headless
ENV SHOW 1

# Script to setup display server & VNC is always executed.
ENTRYPOINT ["docker-entrypoint.sh"]
# Default command to run. This is replaced by appending own command, e.g. `docker run ... node prime-gaming` to only run this script.
CMD node epic-games; node prime-gaming; node gog

@xAffan
Copy link

xAffan commented Jul 17, 2023

just port the script to chrome and use the above projects.

@vogler
Copy link
Owner Author

vogler commented Jul 17, 2023

@xAffan All scripts used Chrome initially, then got captcha challenges (and some other issues) which was resolved by changing to Firefox. See #39.

@xAffan
Copy link

xAffan commented Jul 17, 2023

How about puppeteer stealth plugins?

@ladiko
Copy link

ladiko commented Aug 16, 2023

Just a side note, and maybe not what you want. Why don't you just follow the steps which are described in the Readme.md under "I want to run without Docker or develop locally."

On Windows I just installed Git for Windows, nodejs and python3 (didn't test if it's necessary if I don't need apprise), than ran npm install inside the local repo folder (i think unnecessary as npm update will do it if necessary) and added a batch file which is ran every X hours by task scheduler:

cd /d "%~dp0"

git pull --quiet

@REM by default all browsers (chromium, firefox, webkit) are downloaded
@REM disable browser download during npm update/install
set PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
cmd /c npm update --silent

@REM by default browsers are downloaded to %LOCALAPPDATA%\pw-browsers
@REM set browser download location inside the free-games-claimer directory
@REM run npx in a separate cmd as npx will close the one it is running in
set PLAYWRIGHT_BROWSERS_PATH=%~dp0\pw-browsers
cmd /c npx --quiet playwright install firefox

node epic-games
node gog
node prime-gaming

@REM cleanup
rmdir /s /q "%LOCALAPPDATA%\ms-playwright" "%~dp0\data\screenshots\" "%~dp0\data\browser\cache2" "%~dp0\data\browser\startupCache" "%~dp0\data\browser\storage"
del /f "%~dp0\data\browser\favicons.sqlite" "%~dp0\data\browser\places.sqlite"

All requirements can actually be seen in Dockerfile and Readme.md. Up to you if you want to run it locally or prefer a Docker container, but it won't get much smaller than that.

@vogler
Copy link
Owner Author

vogler commented Aug 17, 2023

How about puppeteer stealth plugins?

@xAffan What about them? The scripts have been using them from the beginning. With the stealth plugins Chrome got some captcha challenges, while Firefox didn't. You can check if the situation changed, but I guess not.

@vogler
Copy link
Owner Author

vogler commented Aug 17, 2023

On Windows I just installed Git for Windows, nodejs and python3 (didn't test if it's necessary if I don't need apprise)

@ladiko python is only needed for apprise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed optimization
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants