Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Upstream Testing #185

Open
wants to merge 10 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
fail-fast: true
matrix:
python-version: ["3.9"]
nautobot-version: ["1.2.0"]
nautobot-version: ["1.5.10"]
env:
INVOKE_NAUTOBOT_VERSION_CONTROL_PYTHON_VER: "${{ matrix.python-version }}"
INVOKE_NAUTOBOT_VERSION_CONTROL_NAUTOBOT_VER: "${{ matrix.nautobot-version }}"
Expand Down Expand Up @@ -126,8 +126,8 @@ jobs:
strategy:
fail-fast: true
matrix:
python-version: ["3.6", "3.7", "3.8", "3.9"]
nautobot-version: ["1.2.0-beta.1"]
python-version: ["3.7", "3.8", "3.9", "3.10"]
nautobot-version: ["1.5.10"]
runs-on: "ubuntu-20.04"
env:
INVOKE_NAUTOBOT_VERSION_CONTROL_PYTHON_VER: "${{ matrix.python-version }}"
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/upstream_testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
name: "Nautobot Upstream Testing"

on: # yamllint disable-line rule:truthy rule:comments
schedule:
- cron: "0 4 */2 * *" # every other day at midnight
push: # yamllint disable rule:empty-values
# TODO: REMOVE PUSH LINE ABOVE
Comment on lines +7 to +8
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
push: # yamllint disable rule:empty-values
# TODO: REMOVE PUSH LINE ABOVE

Remove before merging.


jobs:
upstream-test:
uses: "nautobot/nautobot/.github/workflows/plugin_upstream_testing_base.yml@develop"
with: # Below could potentially be collapsed into a single argument if a concrete relationship between both is enforced
invoke_context_name: "NAUTOBOT_VERSION_CONTROL"
plugin_name: "nautobot-dolt"
82 changes: 70 additions & 12 deletions development/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,20 +1,78 @@
# -------------------------------------------------------------------------------------
# Nautobot App Developement Dockerfile Template
# Version: 1.1.0
#
# Apps that need to add additional steps or packages can do in the section below.
# -------------------------------------------------------------------------------------
# !!! USE CAUTION WHEN MODIFYING LINES BELOW

ARG PYTHON_VER
ARG NAUTOBOT_VER
# Accepts a desired Nautobot version as build argument, default to 1.5
ARG NAUTOBOT_VER="1.5"

# Accepts a desired Python version as build argument, default to 3.8
ARG PYTHON_VER="3.8"

# Retrieve published development image of Nautobot base which should include most CI dependencies
FROM ghcr.io/nautobot/nautobot-dev:${NAUTOBOT_VER}-py${PYTHON_VER}

WORKDIR /source
# Runtime argument and environment setup
ARG NAUTOBOT_ROOT=/opt/nautobot

ENV prometheus_multiproc_dir=/prom_cache
ENV NAUTOBOT_ROOT ${NAUTOBOT_ROOT}

# Install Poetry manually via its installer script;
# We might be using an older version of Nautobot that includes an older version of Poetry
# and CI and local development may have a newer version of Poetry
# Since this is only used for development and we don't ship this container, pinning Poetry back is not expressly necessary
# We also don't need virtual environments in container
RUN curl -sSL https://install.python-poetry.org | python3 - && \
poetry config virtualenvs.create false

# !!! USE CAUTION WHEN MODIFYING LINES ABOVE
# -------------------------------------------------------------------------------------
# App-specifc system build/test dependencies.
#
# Example: LDAP requires `libldap2-dev` to be apt-installed before the Python package.
# -------------------------------------------------------------------------------------
# --> Start safe to modify section

RUN pip install --upgrade pip
# Uncomment the lines below if you are apt-installing any package.
# RUN apt-get -y update && apt-get -y install \
# libldap2-dev \
# && rm -rf /var/lib/apt/lists/*

# Copy in only pyproject.toml/poetry.lock to help with caching this layer if no updates to dependencies
COPY poetry.lock pyproject.toml /source/
# --no-root declares not to install the project package since we're wanting to take advantage of caching dependency installation
# and the project is copied in and installed after this step
RUN poetry install --no-interaction --no-ansi --no-root
# --> Stop safe to modify section
# -------------------------------------------------------------------------------------
# Install Nautobot App
# -------------------------------------------------------------------------------------
# !!! USE CAUTION WHEN MODIFYING LINES BELOW

# Copy in the rest of the source code and install local Nautobot plugin
# Copy in the source code
WORKDIR /source
COPY . /source
RUN poetry install --no-interaction --no-ansi

COPY development/nautobot_config.py /opt/nautobot/nautobot_config.py
# Get container's installed Nautobot version as a forced constraint
# NAUTOBOT_VER may be a branch name and not a published release therefor we need to get the installed version
# so pip can use it to recognize local constraints.
RUN pip show nautobot | grep "^Version: " | sed -e 's/Version: /nautobot==/' > constraints.txt

# Use Poetry to grab dev dependencies from the lock file
# Can be improved in Poetry 1.2 which allows `poetry install --only dev`
#
# We can't use the entire freeze as it takes forever to resolve with rigidly fixed non-direct dependencies,
# especially those that are only direct to Nautobot but the container included versions slightly mismatch
RUN poetry export -f requirements.txt --without-hashes --output poetry_freeze_base.txt
RUN poetry export -f requirements.txt --with dev --without-hashes --output poetry_freeze_all.txt
RUN sort poetry_freeze_base.txt poetry_freeze_all.txt | uniq -u > poetry_freeze_dev.txt

# Install all local project as editable, constrained on Nautobot version, to get any additional
# direct dependencies of the app
RUN pip install -c constraints.txt -e .

# Install any dev dependencies frozen from Poetry
# Can be improved in Poetry 1.2 which allows `poetry install --only dev`
RUN pip install -c constraints.txt -r poetry_freeze_dev.txt

COPY development/nautobot_config.py ${NAUTOBOT_ROOT}/nautobot_config.py
# !!! USE CAUTION WHEN MODIFYING LINES ABOVE
7 changes: 4 additions & 3 deletions development/Dockerfile-dolt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM ubuntu:20.04
FROM ubuntu:22.04

RUN apt update && apt install -y curl mariadb-client && \
apt-get autoremove -y && \
Expand All @@ -7,7 +7,7 @@ RUN apt update && apt install -y curl mariadb-client && \

RUN mkdir -p /var/lib/nautobot

ARG DOLT_RELEASE="v0.35.5"
ARG DOLT_RELEASE="v0.54.2"
RUN curl -L https://github.com/dolthub/dolt/releases/download/${DOLT_RELEASE}/install.sh | bash

RUN dolt config --global --add user.name nautobot
Expand All @@ -24,4 +24,5 @@ ENV DOLT_ENABLE_DB_REVISIONS=true

RUN dolt init

CMD dolt sql-server --config /dolt-config.yaml
ARG DOLT_TRACE=""
CMD dolt sql-server --config /dolt-config.yaml ${DOLT_TRACE}
1 change: 1 addition & 0 deletions development/docker-compose.requirements.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
version: "3.4"
services:
dolt:
platform: "linux/amd64"
build:
context: "../"
dockerfile: "development/Dockerfile-dolt"
Expand Down
6 changes: 3 additions & 3 deletions nautobot_version_control/diffs.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,10 +116,10 @@ def diff_summary_for_table(table, from_commit, to_commit):
"removed": 0,
}
with connection.cursor() as cursor:
cursor.execute(
f"""SELECT diff_type, count(diff_type) FROM dolt_commit_diff_{table} # nosec
cursor.execute( # nosec
f"""SELECT diff_type, count(diff_type) FROM dolt_commit_diff_{table}
WHERE to_commit = %s AND from_commit = %s
GROUP BY diff_type ORDER BY diff_type""", # nosec
GROUP BY diff_type ORDER BY diff_type""",
(to_commit, from_commit),
)
for diff_type, count in cursor.fetchall():
Expand Down
2 changes: 2 additions & 0 deletions nautobot_version_control/filters.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ class Meta:
"latest_commit_date",
"latest_commit_message",
)
exclude = ["id"]

def search(self, queryset, name, value): # pylint: disable=unused-argument,no-self-use
"""
Expand Down Expand Up @@ -64,6 +65,7 @@ class Meta:
"date",
"message",
)
exclude = ["id"]

def search(self, queryset, name, value): # pylint: disable=unused-argument,no-self-use
"""
Expand Down
1 change: 0 additions & 1 deletion nautobot_version_control/migrations/0001_initial.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@


class Migration(migrations.Migration):

initial = True

dependencies = []
Expand Down
3 changes: 1 addition & 2 deletions nautobot_version_control/migrations/0002_branchmeta.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("nautobot_version_control", "0001_initial"),
Expand All @@ -16,7 +15,7 @@ class Migration(migrations.Migration):
migrations.CreateModel(
name="BranchMeta",
fields=[
("branch", models.TextField(primary_key=True, serialize=False)),
("branch", models.CharField(max_length=240, primary_key=True, serialize=False)),
("source_branch", models.TextField()),
("created", models.DateTimeField(auto_now_add=True, null=True)),
(
Expand Down
1 change: 0 additions & 1 deletion nautobot_version_control/migrations/0003_conflicts.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [
("nautobot_version_control", "0002_branchmeta"),
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [
("nautobot_version_control", "0003_conflicts"),
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("nautobot_version_control", "0004_constraintviolations"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@


class Migration(migrations.Migration):

dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("nautobot_version_control", "0005_pullrequest_pullrequestreview"),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@


class Migration(migrations.Migration):

dependencies = [
("nautobot_version_control", "0006_auto_20210817_0251"),
]
Expand Down
Loading