Added all open source licenses (Fixes #2941) #3072

Andrew-Chen-Wang · 2021-02-27T05:28:34Z

Description

Use licenses that we see on GitHub when we create a new repo. Also included is automatic updating of the licenses from the aforementioned license repository created by GitHub. The cookiecutter.json file is also updated.

Uses open source licenses (including the Jekyll format) from https://github.com/github/choosealicense.com/tree/gh-pages/_licenses
Added license section in README.rst because some licenses want it to be explicit
cookiecutter.json shows all licenses (including "Not open source") in lexicographical order except for "Not open source"

Checklist:

I've made sure that tests/test_cookiecutter_generation.py is updated accordingly (especially if adding or updating a template option)
I've updated the documentation or confirm that my change doesn't require any updates

Docs haven't been updated because honestly... I don't know if we want to show ALL of those licenses...

Rationale

Fixes #2941

Lots of open source licenses... too many to choose.

Makes it easier to maintain since we can just use GitHub actions to constantly update the files if necessary.

* Uses open source licenses (including the Jekyll format) from https://github.com/github/choosealicense.com/tree/gh-pages/_licenses * Added license section in README.rst because some licenses want it to be explicit * cookiecutter.json shows all licenses (including "Not open source") in lexicographical order except for "Not open source"

Signed-off-by: Andrew-Chen-Wang <[email protected]>

Andrew-Chen-Wang · 2021-02-27T05:49:48Z

Oooo that's an... interesting error to deal with...

Signed-off-by: Andrew-Chen-Wang <[email protected]>

* The solution? Put it in some random txt file in the license folder and read it in. You can't use """""" or "" or '' without some kind of error. By reading in the text instead, we completely avoid this. Signed-off-by: Andrew-Chen-Wang <[email protected]>

Andrew-Chen-Wang · 2021-02-27T06:54:17Z

Something that can make sure commits decrease in the automatic workflow is to check if the license file already exists since licenses don't change due to their copyright reasons. That means we should only commit if there's a difference in cookiecutter.json (i.e. any new license).

Additionally, it might be better to also include the MIT license, Apache, etc. near the front like "Not open source" license is in cookiecutter.json.

Andrew-Chen-Wang · 2021-02-27T16:36:37Z

Just realized we should instead make a pull request instead of committing directly in case of a new file since there would be square brackets.

* Deleted LICENSE and COPYING files * Moved most utilized options to front of licenses in cookiecutter.json options. Signed-off-by: Andrew-Chen-Wang <[email protected]>

Andrew-Chen-Wang

Perhaps I missed a couple. I think that should be it from my initial generation (this was before I actually programmed the autoupdater)

Andrew-Chen-Wang · 2021-02-27T20:57:09Z

{{cookiecutter.project_slug}}/licenses/bsd-4-clause.txt

+
+3. All advertising materials mentioning features or use of this software must
+   display the following acknowledgement:
+     This product includes software developed by [project].


Suggested change

This product includes software developed by [project].

This product includes software developed by {{cookiecutter.project_name}}.

Andrew-Chen-Wang · 2021-02-27T20:59:32Z

{{cookiecutter.project_slug}}/licenses/apache-2.0.txt

+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]


Suggested change

Copyright [yyyy] [name of copyright owner]

Copyright {% now 'utc', '%Y' %} {{cookiecutter.author_name}}

browniebroke

Just realized we should instead make a pull request instead of committing directly in case of a new file since there would be square brackets.

We can reuse what we do to keep pre-commit hooks config up to date:

https://github.com/pydanny/cookiecutter-django/blob/master/.github/workflows/pre-commit-autoupdate.yml

It doesn't do anything in case of no changes.

browniebroke · 2021-03-02T21:29:11Z

hooks/post_gen_project.py

@@ -324,6 +312,34 @@ def remove_storages_module():
    os.remove(os.path.join("{{cookiecutter.project_slug}}", "utils", "storages.py"))


+def handle_licenses():


Is this to remove the Jekyll front matter? I'm wondering if wouldn't make more sense to do this work in the update_licenses.py scripts maybe?

We can afford to use more modern Python there, we control the Python version better.

Sorta, we need the Jekyll formatting at first. Each file's name is not the same as the title. So in order to find the correct file, we loop through each file until we find one that has the Jekyll title of the exact license name from cookiecutter.json.

We could improve on this inefficient method though by having update_licenses.py update this Python file to have a dictionary that maps titles (the dict keys) to their files (the dict values). That way we can remove the need for the Jekyll headers.

Ah, I see...

browniebroke · 2021-03-02T21:32:30Z

scripts/update_licenses.py

+    with open("cookiecutter.json") as f:
+        data = json.load(f)
+    data["open_source_license"] = titles
+    with open("cookiecutter.json", "wt") as f:
+        json.dump(data, f, indent=2)


You can do away without the context manager with Pathlib's read_text() and write_text() APIs. Not a big deal, I find them a bit higher level.

browniebroke · 2021-03-02T21:34:25Z

scripts/update_licenses.py

+year = (re.compile(r"\[year]"), "{% now 'utc', '%Y' %}")
+email = (re.compile(r"\[email]"), "{{ cookiecutter.email }}")
+fullname = (re.compile(r"\[fullname]"), "{{ cookiecutter.author_name }}")
+project = (re.compile(r"\[project]"), "{{ cookiecutter.project_name }}")
+projecturl = (re.compile(r"\[projecturl]"), "{{ cookiecutter.domain_name }}")


Do you think we could add a test to make sure we don't miss any new placeholder? Seems a bit tricky to make the difference between a placeholder and a genuine [...] though. 🤔

That's why I wanted the pull request initially... In the Jekyll headers for all these files, there are directions that specify what to replace though! The directions say something like "replace [year] with etc." if I'm not mistaken. So after replacing everything, if we still match a regex of \[.*?\] (which matches to anything with square brackets including if there are multiple square brackets like [asdf] [asf] will be two matches).

Side note: I think we don't really need to care about Appendix directions (like that found in Apache 2.0 where they say to add the boilerplate code with year and name. With us putting it in already, I think the developer will understand. Additionally, license years don't have to be updated so there's that...)

browniebroke · 2021-03-02T21:35:50Z

cookiecutter.json

-    "Apache Software License 2.0",
-    "Not open source"
-  ],
+  "open_source_license": ["Not open source", "MIT License", "BSD 3-Clause \"New\" or \"Revised\" License", "GNU General Public License v3.0", "Apache License 2.0", "\"Do What The F*ck You Want To Public License\"", "Academic Free License v3.0", "Artistic License 2.0", "BSD 2-Clause \"Simplified\" License", "BSD 3-Clause Clear License", "BSD 4-Clause \"Original\" or \"Old\" License", "BSD Zero Clause License", "Boost Software License 1.0", "CeCILL Free Software License Agreement v2.1", "Creative Commons Attribution 4.0 International", "Creative Commons Attribution Share Alike 4.0 International", "Creative Commons Zero v1.0 Universal", "Eclipse Public License 1.0", "Eclipse Public License 2.0", "Educational Community License v2.0", "European Union Public License 1.1", "European Union Public License 1.2", "GNU Affero General Public License v3.0", "GNU General Public License v2.0", "GNU Lesser General Public License v2.1", "GNU Lesser General Public License v3.0", "ISC License", "LaTeX Project Public License v1.3c", "Microsoft Public License", "Microsoft Reciprocal License", "Mozilla Public License 2.0", "ODC Open Database License v1.0", "Open Software License 3.0", "PostgreSQL License", "SIL Open Font License 1.1", "The Unlicense", "Universal Permissive License v1.0", "University of Illinois/NCSA Open Source License", "Vim License", "zlib License"],


I see the script has now indenting this properly, although this is not yet reflected here.

I'm slightly confused what you mean here? Are you saying that the GitHub workflow may not get the indentation correctly? Hopefully when the argument "indent=2" or 4 (can't remember) from json.dump works properly (preserving ordering as well). But I can't remember during my test run if all the arrays became... well aren't vertically spaced out like postgresql_version

With indent=2, I would expect this part to wrap vertically, a bit like it used to, but it doesn't. I was trying to ask if it's because you generated this with an earlier version where indent was missing or if my expectation is just wrong.

(sorry looking back, my initial comment was poorly worded)

browniebroke · 2021-03-02T21:37:33Z

.github/workflows/update-licenses.yml

+  # Every day at 2am
+  schedule:
+    - cron: "0 2 * * *"


We might want to run this at another time that the chnagelog update.

Andrew-Chen-Wang · 2021-07-27T09:25:51Z

@browniebroke I'll be creating a new PR at some point in the long distance future. Looks like I completely missed their (new?) API that includes the licenses: https://docs.github.com/en/rest/reference/licenses#get-a-license

Andrew-Chen-Wang · 2022-12-12T01:44:29Z

#3982 would resolve this

Andrew-Chen-Wang added 3 commits February 27, 2021 00:22

Fix README using incorrect variable

c377de0

Signed-off-by: Andrew-Chen-Wang <[email protected]>

Fix license dir location in post_gen hook

0d42cfb

Signed-off-by: Andrew-Chen-Wang <[email protected]>

Andrew-Chen-Wang added 2 commits February 27, 2021 01:23

Add automatic update license workflow

a33be2a

Signed-off-by: Andrew-Chen-Wang <[email protected]>

Andrew-Chen-Wang force-pushed the licenses branch from 36f0687 to 8666291 Compare February 27, 2021 17:10

Autoupdate replace Jekyll variables for Jina

13459e4

* Deleted LICENSE and COPYING files * Moved most utilized options to front of licenses in cookiecutter.json options. Signed-off-by: Andrew-Chen-Wang <[email protected]>

Andrew-Chen-Wang commented Feb 27, 2021

View reviewed changes

browniebroke reviewed Mar 2, 2021

View reviewed changes

Andrew-Chen-Wang closed this Jul 27, 2021

Andrew-Chen-Wang reopened this Jul 27, 2021

alexzvk mentioned this pull request Dec 6, 2022

Adds open source licenses (fixes #2941) #3982

Open

2 tasks

Andrew-Chen-Wang closed this Dec 12, 2022

Andrew-Chen-Wang deleted the licenses branch December 12, 2022 01:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added all open source licenses (Fixes #2941) #3072

Added all open source licenses (Fixes #2941) #3072

Andrew-Chen-Wang commented Feb 27, 2021 •

edited

Loading

Andrew-Chen-Wang commented Feb 27, 2021

Andrew-Chen-Wang commented Feb 27, 2021

Andrew-Chen-Wang commented Feb 27, 2021

Andrew-Chen-Wang left a comment

Andrew-Chen-Wang Feb 27, 2021

Andrew-Chen-Wang Feb 27, 2021

browniebroke left a comment

browniebroke Mar 2, 2021

Andrew-Chen-Wang Mar 2, 2021

browniebroke Mar 2, 2021

browniebroke Mar 2, 2021

browniebroke Mar 2, 2021

Andrew-Chen-Wang Mar 2, 2021

browniebroke Mar 2, 2021

Andrew-Chen-Wang Mar 2, 2021

browniebroke Mar 2, 2021 •

edited

Loading

browniebroke Mar 2, 2021

Andrew-Chen-Wang commented Jul 27, 2021 •

edited

Loading

Andrew-Chen-Wang commented Dec 12, 2022

	This product includes software developed by [project].
	This product includes software developed by {{cookiecutter.project_name}}.

	Copyright [yyyy] [name of copyright owner]
	Copyright {% now 'utc', '%Y' %} {{cookiecutter.author_name}}

		@@ -324,6 +312,34 @@ def remove_storages_module():
		os.remove(os.path.join("{{cookiecutter.project_slug}}", "utils", "storages.py"))


		def handle_licenses():

Added all open source licenses (Fixes #2941) #3072

Added all open source licenses (Fixes #2941) #3072

Conversation

Andrew-Chen-Wang commented Feb 27, 2021 • edited Loading

Description

Rationale

Andrew-Chen-Wang commented Feb 27, 2021

Andrew-Chen-Wang commented Feb 27, 2021

Andrew-Chen-Wang commented Feb 27, 2021

Andrew-Chen-Wang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

browniebroke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

browniebroke Mar 2, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Andrew-Chen-Wang commented Jul 27, 2021 • edited Loading

Andrew-Chen-Wang commented Dec 12, 2022

Andrew-Chen-Wang commented Feb 27, 2021 •

edited

Loading

browniebroke Mar 2, 2021 •

edited

Loading

Andrew-Chen-Wang commented Jul 27, 2021 •

edited

Loading