A LOT has been done since the last release. But this will be another point on time version that, again, for this moment will be just a list of the commits. Old documented features still work and maybe the older conservative approach would be what becomes implementer instead of what will be commented here.
I will comment about something drafted called HDPLisp.
1. About HDPLisp (temporary name)
I think this feature alone may take at least 150 hours to have a general idea of if this doing such an approach could work. What really push to the limit (to try usability even on the auxiliary script language to HDP) was the following:
something safer enough to (as extreme, but realistic example at this point from what I'm seeing in the field) be able to allow be used by who work on very frontline to replace (Excel/Google Spreadsheet) formulas on contexts like even ones used decide people who would not have access to Intensive Care Units.
I avoid talking about this type of thing, but I'm acknowledging that theoretical decisions from last year based on ethicists and medical associations on CPLP (including how other countries would deal with lack of ICUs to everyone) are challenging. Software broke, like overflow beyond 100% because underlying data is bad (and data is bad because it reflects really). Even if HDP was ready years ago, with a level of automation we here would abstract (and already in local languages) either medical personnel would be able to fix things following these ethical codes or people with programming skills would need to fix the IF-ELSEs even when who make the requests make mistakes.
1.1 Hypothesis: something that (if necessary) be as strict as Ada, yet as feasible to implement compilers, like Lisp
Of all programming languages I researched for inspiration, Ada is the closest one that seems to try to solve issues that HDP would need extensions for extreme cases were mistakes cost lives. Ada use of English and (in special consideration alternatives at time was created) the way to reduce ambiguity (it only has + - / * as mathematical operators) is an approach to try to be more explicit (so, even if we make something compatible with Lisp/Scheme/Clojure, the real working version could be very verbose, to a point of people could write a draft in middle of an in person reunion). Also the "Programming by contract" / Design by contract (DbC), is a new feature and ships with core version of Ada 2012, is something that could be very, very pertinent to use cases of HDPLisp, to a point this alone could take more time on next weeks.
The decision to try to be more close to a Lisp is because, is the closest to a Turing-complete alternative feasible to implement as soon as possible focusing on being production-ready starting from scratch. This does not means in 150-300 hours may not just be more restricted on how extensions to HDP may be (like put the conditions on the HDP files themselves, just a different file extension) but all other typical programming languages architectural approaches even if we manage to make something work in a single natural language, how the verbs could be translated? And did anyone ever planned a fully translatable turing-complete language that the parser is able to deal with both Left-to-right (like Latin) or Right-To-Left (like Arab script)?
What I mean is that the decision about a more S-expression as general approach to the auxiliary language of HDP takes less time to implement a text that can be "converted to commands" (Abstract syntax tree) (even if would not be for full support for several natural languages languages). This time starts to get very pertinent if have to port what would be HDPLisp to more than one hosting language (think not only python, but full support with JavaScript, so both browser and some some desktop application could be done). Just to give an idea: the bare minimum syntax of Lisp-like sintaxes are know to be able to run on very restricted hardware; which means that any subset of what would be HDPLisp (like the ones that would not be used for abstract complex things that may need more frequent update, like access to UN P-codes) could be ported to specialized, cheap to build (yet feasible to be audited) hardware that could not even need to have an operational system.
2. git log --oneline -172
bd28da2 v0.8.6 started; (we may use hxlm-js (#16) + hdplisp (#18) even on python port to do heavy processing of HDPLisp until API be stable
2f60059 hdplisp (#18): added several examples from other lisps (and non-lisps, like Ada and Haskell)
1cb8de3 hxlm-js (#16), hdplisp (#18): I think lisp/norvig-lispy.mjs (based on Peter v1) will not be engouth; I think we could refactor all the things and already plan ahead 'programming by contract' already on the prototypes of functions when user create them
e312774 hxlm-js (#16), hdplisp (#18): LISP-1 or LISP-2? That's a good question
e610e0d hxlm-js (#16), hdplisp (#18): HDLbLispMachinamSimulatum draft
75f426f hxlm-js (#16), hdplisp (#18): hdplisp-editor-ltr.html testing xtermjs
2747d33 hxlm-js (#16), hdplisp (#18): hdplisp-editor-ltr.html draft
4007e91 hxlm-js (#16), hdplisp (#18): NodeJS REPL ; TODO: we need safer ways to do it
49d2e2f hxlm-js (#16), hdplisp (#18): HDPbMiniman.machinam_simulatum() draft
cd4c930 hxlm-js (#18), hdplisp (#18): testing extension .hdpl instead of .hdpl.lisp; Uses github/linguist
306b076 hxlm-js (#18), hdplisp (#18): learned identicum? non-identicum?
cbd6c03 hxlm-js (#18), hdplisp (#18): lat->multiplicationem, lat->divisionem
5f3c999 hxlm-js (#18), hdplisp (#18): Improved naming references of the boostrapper
52fc4d4 hxlm-js (#18), hdplisp (#18): GREAT! Now + - work as proof of conceptgit gui!
9721bfb hxlm-js (#18): HDPbLisp.evaluate() draft started
a452486 hxlm-js (lisp #18): hdpl-conventions; more more context; some formating
1b03847 hxlm-js (lisp #18): hdpl-conventions; more details about Internationalized auditability core feature
b9900d9 hxlm-js (lisp #18): hdpl-conventions draft
253d631 hxlm-js (lisp #18): moved code based on Peter Norvig work to dedicated file; also some refactoring
24ad442 hxlm-js (lisp #18): directory reorganization
c15b51c hxlm-js (lisp #18): tests with Right-To-Left Mark #15
d0e2063 hxlm-js (lisp #18): HDPbLisp.{ast,ast_ltr,ast_rtl}() started; Right-to-left? (like Imperial Aramaic, not Lingua Latina) will need some serious work on abstract syntax tree
1270c51 hxlm-js (lisp #18): hdpb-lisp.mjs, ok, NOW at least the reader create an LTR Abstract Syntax Tree
ce20548 hxlm-js (lisp #18): hdpb-lisp.mjs, working on parse_recursive_ltr()...
0c5eaca hxlm-js (lisp #18): hdpb-lisp.mjs less loops, more recursion...
4955974 hxlm-js (lisp #18): قلب.apc.hdpl.lisp added
1ecbfde hxlm-js (lisp #18): hdpb-lisp.mjs loops (not yet a lot, but loops; still need to convert to recursion or something)
b50ec4d hxlm-js (lisp #18): ontologia/core.hdplisp.yml draft
0634d31 hxlm-js (hdp #18, lisp #18): prefix HDPb instead of HDP; see https://github.com/EticaAI/HXL-Data-Science-file-formats/issues/18#issuecomment-813734342
ccd131c hxlm-js (hdp #18, lisp #18): HDPL10n & HDPi18n
73ac547 hxlm-js (hdp #18, lisp #18): starting going JavaScript modules all the way (but no node_modules/, not now, let's use native things)
df4b7ee hdp-spec (#16): draft of hdp-conventions/security-considerations.md
a64c37c hdp-integrity (#17): improved prepare-hxlm-relsease.sh (#12) to generate Subresource Integrity (still not fully automated)
f47b97f v0.8.5 started; hxlm-js (hdp #18, lisp #18), the javascript version _starts_ to undestand HDP; part of the code inspired on https://tags.etica.ai
9e20848 hxlm-js (hdp #18, lisp #18) interface start to be usable for demonstrations
8368fe0 hxlm-js (hdp #18, lisp #18) fixed JS promise bug
7bbef57 hxlm-js (hdp #18, lisp #18) tests with gpg-sign (#12); fixed usage of page-signer.js (needs '%%%SIGNED_PAGES_PGP_SIGNATURE%%%' on the source page); also changed order of prepare-hxlm-relsease.sh commands
01af024 hxlm-js (hdp #18, lisp #18) tests with gpg-sign (#12); tests using @tasn webext-signed-pages page-signer.js utility
e8886a4 hxlm-js (hdp #18, lisp #18): index.html also part of the hashes
e05f84a hxlm-js (hdp #18, lisp #18): HDPAux tests
68de75b hxlm-js (hdp #18, lisp #18): Javascript async, await, etc, etc, etc
a644c08 hxlm-js (hdp #18, lisp #18): HDPAux started
4fcbeab #12 prepare-hxlm-relsease.sh created
2d9f678 hxlm-js (hdp #18, lisp #18): HDPMiniman.bootstrapping()
6d4f5a3 hxlm-js (hdp #18, lisp #18): hxlm-js; Great! Definitely modern javascript is much beautiful (and not as complicated to convert from python)
04777d8 hxlm-js (hdp #18, lisp #18): started hxlm-js/; transpile Python to JS is way too ugly to not do by hand
8a208e5 hdp.etica.ai as domain for testing JavaScript early prototypes; also {{ escape with \ as quick fix to allow generate GitHub pages
f1c13e6 hxlm.ontologia.json files generated from hxlm.ontologia yml files; draft of hxlm.core.localization.hgettext
28e5abb hdp-spec (#16): copy draft of hxlm_factum_to_sexpr back to hdp.util.debug
27d30bb hxlm_minimam.lang: hxlm_factum_to_sexpr(), _s(), and draft of _sv() & hxlm_sespr_to_object()
ceddfa6 WOW! Ok. Nice to know. hxlm_minimam.lang draft
db34830 Code refactoring: hxlm.ontologia.python.{commune,systema} created from hxlm.core
7ce4f0d hdp-spec (#16): AbstAttr, AbstAux & AbstRadix created
37f0348 tests/transpile-python-to-javascript.sh
a804249 Code refactoring: hxlm.core.hdp.data -> hxlm.ontologia.python; the idea here is allow even non-python programmers have an idea of how internals of HXLm submodules works
21d72ee Code refactoring: hxlm.ontology -> hxlm.ontologia; the first usage ('Ogdoas scholastica', 1606, Jacob Lorhard, not classical Latin, but Neo-Latin) uses 'ontologia'
1513198 Code refactoring: hxlm.core.hdp.data; sharing responsability with smaller modules
3d97d0e Code refactoring: hxlm.core.hdp.{datamodel.py => data/radix.py}
8fd5044 Code refactoring: hxlm.core.{util.py => util/common.py}
79c43b4 hxlm.lisp drafts: hxlm.lisp.HDPlisp dataclass drafted
f81cacc hxlm.lisp drafts: explain(), parser(), reader(), tokenizer()
c72dada hxlm.lisp started; Also i10n_factum() from i18n+l10n (#15)
3f5ca5f i18n+l10n (#15), hxp-spec (#16): hxlm.core.hdp.datamodel.Factum created; the idea is abstract ram text messages to allow future l10n
e9e281b hdp-spec (#16): hxlm.core.hdp.datamodel.HDPOkay created
fbbba00 i18n+l10n (#15), hxp-spec (#16): hxlm.core.hdp.exception package already have an draft to allow localized exceptions on the user language! (ok that we may need to make it stable engouth to not trow exceptions on exceptions themselves)
b637553 hdp-spec (#16): HDPExceptionem created; when no VKG is availible, we will fallback to Latin
bd37909 hdp-spec (#16): hxlm.core.hdp.datamodel.HDPProjectInfo() created
e937b1b i18n+l10n (#15): VKG att.okay added (seems easy to make localization; wikipedia describes as 'It has been described as the most frequently spoken or written word on the planet' and we need some way to express result of program that is not bad, even if not as perfect)
d818399 HXlm._DT (#5): inicial draft as internal HXLm library
0a16b09 hdp-integrity (#17), hdp-spec (#16): is_not_acceptable_load_this() created
fc29c96 hdp-integrity (#17), hdp-spec (#16): drafted hxlm.core.hdp.hazmat.policy.{_get_user_know_what_is_doing(),_get_bunker()}
c2c9db0 hdp-integrity (#17), hdp-spec (#16): HDPPolicyLoad dataclass improved
4db8ab7 hdp-integrity (#17), hdp-spec (#16): started hxlm.core.hdp.hazmat; HDPPolicyLoad ( and ...hasmat.policy) affects #17, the rest is mostly for the #16)
3d2ba52 hxl-spec (#16) hxlm.core.hdp.project._recursive_resource_parsing() draft
1306d6c New temporary name the python package: 'hdp-toolchain'; was 'hxlm-base-eticaai'
6b81b89 hdp-spec (#16): hxlm.core.hdp.index.convert_resource_to_hdpindex() created
3a091a7 hpd-spec (#16): draft of HDPIndex, HDATUM_EXEMPLUM, hxlm.core.hdp.index, hxlm.core.hdp.item
0849da0 hpd-spec (#16): HDPProject._parse_resource() draft
907830b hpd-spec (#16): Added 'Valid cases for RAW HDP files'; also MVP (minimal viable product) of hxlm.core.hdp.raw
706c93b hpd-spec (#16): hxlm.core.hdp.raw.is_hsilo_like() renamed to is_raw_hdp_item_syntax(); created get_raw_hmeta()
d160997 hpd-spec (#16): hxlm.core.hdp.raw.is_hsilo_like() created
b281107 hpd-spec (#16): hxlm/core/localization/hdp.py -> hxlm/core/hdp/util.py | hxlm/core/hdp/raw.py contains old hxlm/core/hdp/util.py
ca5783e hpd-spec (#16): hxlm.core.hdp.util.ResourceWrapper2HDPRaw() started
6ad03ca hpd-spec (#16): 'hdp-toolchain' over 'hxlm-base-eticaai' could be a good naming to consider when installing? Also clean ups
1935199 v0.8.4 started
fa0b391 hxl-spec (#16): HDPProject._parse_entrypoint() started
6da18a6 hxl-spec (#16): Somewhat acceptable MVP of hxlm.core.io
951e7e9 hxl-spec (#16): hxlm.core.io.net.load_remote_file() created; hxlm.core.io.util.get_entrypoint() already works for EntryPointType.HTTP (exact file only, directory not implemented... yet)
bf23058 hxl-spec (#16): hxlm.core.io.local.load_any_file() created; hxlm.core.io.util.get_entrypoint() already works for EntryPointType.LOCAL_DIR
f720342 hxl-spec (#16): hxlm.core.io.util.get_entrypoint() already works for EntryPointType.LOCAL_FILE
d443f0e hxl-spec (#16): hxlm.core.io.util._get_infered_filenames() created
c8b98fb hxl-spec (#16): Contants now by default should have file://host/ prefix when refering to local path; hxlm.core.io.util.strip_file_protocol() created;
85ea632 hxl-spec (#16): hxlm.core.io (dedicated I/O related utils) created; content either moved from other places or mere early draft
3b25ebd Oops.. explicitly removing 'Copyleft 🄯' (conflict with Public Domain)
e270e4a hxl-spec (#16): hxlm.core.hdp.datamodel started; HDPRaw, AttrAdm0, AttrDescriptionem, HDatumItem, HFilumItem, HTransformareItem, HMetaItem, HMetaWrapper, HSiloItem, HSiloWrapper
3bf0a3a i18n+l10n (#15), hxl-spec (#16): L10NContext, ResourceWrapper created
0d137a9 hxl-spec (#16): hxlm.ontology.core.urn.yml draft created
7c1d701 hxl-spec (#16): hxlm.core.util.get_entrypoint_type() done
9256978 hxl-spec (#16): hxlm.core.util.get_entrypoint_type() started; hxlm.core.types.RemoteType Enum started
b669bf3 hxl-spec (#16): hxlm.core.types created; moved hxl.core.hdp.py to subfolder
d45459f hxl-spec (#16): .hdp.yml file initial draft
de9b552 hxl-spec (#16): HXLm.HDP.check_integrity() mininal viable product
133695b hxl-spec (#16): HXLm.HDP._get_checksum_keyterm() created; HXLm.HDP.check_integrity() drafted
bf537f6 i18n+l10n (#15): Core vkg.numerum initial draft (see https://github.com/EticaAI/HXL-Data-Science-file-formats/issues/15#issuecomment-808095624)
cbf4333 i18n+l10n (#15), hxl-spec (#16): HXLm.HDP.checksum() created
812113b i18n+l10n (#15), hxl-spec (#16): HXLm.HDP.hashable() created
70b8c18 i18n+l10n (#15), hxl-spec (#16): HXLm.HDP.transpose() now have an rudimentar way to automatically add headers when transposing
69cd64c i18n+l10n (#15), hxl-spec (#16): HXLm.HDP._clean_metakeys() created
35b9dcc i18n+l10n (#15), hxl-spec (#16): HXLm.HDP.transpose() add some metadata with <<>> to make viable process objects
144e2ad i18n+l10n (#15), hxl-spec (#16): core.lvk.yml: fs.hdp.base added; now is possible to select order of files as default
0cf6609 hxl-spec (#16): HDATUM_HXL (hxl.eng.hdp.yml, HXL Standard) data sets added as top level constant
459624d hxl-spec (#16): HXLm.HDP.load() created
533d3b4 automated-testing (#12): test_pytest_doctest_modules_all_may_have_false_positives() created; this is ugly, but may be able to allow run 'pytest -vv hxlm/ --doctest-modules'-like command even if pytest explicitly does not support; see https://github.com/pytest-dev/pytest/issues/2042#issuecomment-381309723
df0e5bb hxl-spec (#16): HXLm.HDP namespace draft
a47f9f9 hdp-spec (#16): HXLm.L10N.get_metadata() created
b2be871 hdp-spec (#16): created HXLm.L10N.get_hsilo_checksum() & hxlm.core.internal.integrity.get_checksum_crc32()
da065bf i18n+l10n (#15), hdp-spec (#16): get_meta_header() created
93fc3f2 v0.8.3: HDP/hxlm already allow any know language be a reference one :,)
40ca452 i18n+l10n (#15), hdp-spec (#16): _ALMOST_ on the fly go back from any language; already is possible to LAT->RUS->POR->(...) but without hashing, the header could grown exponentialy; BUT we're almost near each language be equivalent!
33f0663 i18n+l10n (#15), hdp-spec (#16): hxlm.ontology README added
ef50fe0 i18n+l10n (#15), hdp-spec (#16): syntatic sugar for using HXLm directly as importable library
14600e8 i18n+l10n (#15), hdp-spec (#16): hxlm.ontology started (based on hxlm/core/localization/core_loc.yml,hxlm/core/schema/core_vocab.yml & hxlm/core/schema/hdp.json-schema.json; (this will break a bit the automated testing\!)
057f06e i18n+l10n (#15): build_new_vocabulary_knowledge_graph MVP
5330899 i18n+l10n (#15): build_new_vocabulary_knowledge_graph started
41ea4aa i18n+l10n (#15): Added concept of Active VKG (Vocabulary Knowledge Graph) ; See https://github.com/EticaAI/HXL-Data-Science-file-formats/issues/15#issuecomment-804535992
5e1475f hdp-spec (#16): MVP of refactoring (part of the functionality only availible internally on HDP class, not is possible with direct access via functions)
68135c9 hdp-spec (#16): refactoring started
eccd959 i18n+l10n (#15): hxml.data.udhr: created, inspired on the FANTASTIC project from unicode-org, https://unicode.org/udhr/!
6c79367 hdp-spec (#16), hdp-integrity (#17): created _get_hash_blake2() & _get_hash_sha3_512(); _get_hash_sha2_256() now raises warning and suggest SHA-3 or _get_hash_sha2_224(); because the SHA-2 224 actually in some cases is more resistant than SHA-2 256
8af1d5e hdp-spec (#16), hdp-integrity (#17): created _get_hash_sha3_256(); also several comments about hash colision attacks
20f88c0 hdp-spec (#16), hdp-integrity (#17): created _get_hash_md5() & _get_hash_sha2_256()
c4b19d2 hdp-spec (#16), hdp-integrity (#17): started hxlm.core.internal.integrity (was ...signature)
dce6fab hdp-spec (#16), hxlm (#11): hxlm.core.internal.signature started
2b17839 hdp-spec (#16), i18n+l10n (#15): get_language_from_hdp_raw() works; created hxlm.core.util.load_file() (generic file loader, useful for tests)
c90f216 hdp-spec (#16), i18n+l10n (#15): get_lid_from_keyterm() works
909c387 hdp-spec (#16), i18n+l10n (#15): some function drafts; also we're using ISO 369-3 + ISO 15924 as lid code (just ISO 369-3 does not work for languages with several scripts
a09825b hdp-spec (#16), i18n+l10n (#15): moved lid ('localization ID') from core_voc.yml to core_loc.yml
84f7175 hdp-spec (#16), i18n+l10n (#15): thanks @kahsieh for the https://www.kevinhsieh.net/2019/02/27/chinese-macrolanguage/! And yes, we at least are allowing expand later for more than 2 variants; added core_vocab.lid FRA, POR, RUS, ZHO;
5096dc0 hdp-spec (#16), i18n+l10n (#15): started core_vocab.lid (early tests with use of languages names for thenselves already with their original script (this could allow also detect both language AND script; see https://github.com/EticaAI/HXL-Data-Science-file-formats/issues/16#issuecomment-803493516)
5aac3ec hdp-spec (#16): hdp-conventions, added 'Tokens meaning on keys'
caf9c94 hdp-spec (#16): hdp-conventions initial draft
224255e hdp-spec (#16): ( ) [ ] { } < > tests on keys; see https://github.com/EticaAI/HXL-Data-Science-file-formats/issues/16#issuecomment-803246739
72d2513 v0.8.2 started; using hdpcli now by default will show in version based on user preference languages (uses enviroment variables, see i18n+l10n (#15))
e8c9a1f hdp-spec (#16), i18n+l10n (#15): hxlm.core.localization.translation module draft
549f196 i18n+l10n (#15): MVP of localization subpackage!
3565eea i18n+l10n (#15): l10n get_localization_knowledge_graph() created
91c6d79 i18n+l10n (#15): Oh boy, this will be fantastic; Created core_loc.yml an drafted hxlm.core.localization.util.get_language_preferred()
8ae8fc9 i18n+l10n (#15): hxlm.core.localization started
d9dc16a hdp-spec (#16), i18n+l10n (#15): HVocabHelper().get_languages_of_hsilo() is somewhat already usable
b08bcf1 hdp-spec (#16), i18n+l10n (#15): HVocabHelper()._parse_language_score_details() created
5f1ef9a hdp-spec (#16), i18n+l10n (#15): more loops; HVocabHelper().get_languages_of_words() is almost there...
1739b60 hdp-spec (#16), i18n+l10n (#15): I see loops... all the time. They are everywhere
1486ac3 hdp-spec (#16), i18n+l10n (#15): drafted HVocabHelper get_languages_of_hsilo() & get_languages_of_words()
60e715d hdp-spec (#16), i18n+l10n (#15): HVocabHelper().get_languages_of_vocab() created
86db83e automated-testing (#12), hdp (#16): added doctest to hxlm.core.schema.vocab (good to know that is possible to add some sort of test even on an class or method itself
9f09a4e hdp-spec (#16), i18n+l10n (#15): HVocabHelper.get_translation_value() created; hdpcli is already doing too much; lets break functionlity more on other parts of hxlm lib
2647e7c automated-testing (#12): improved ways to check scripts directly
e91e2f9 automated-testing (#12), i18n+l10n (#15): added test-infra to be able to check EVEN command line instead of only the underlinign library; also removed nacl dependency (it was required for run the hdpcli, but it was not tested until now on a clean enviroment)
70ba77c i18n+l10n (#15): thats fantastic! So is possible to translate _even_ command line interface
7a0a00b i18n+l10n (#15): gettext tests
a8030d3 hdp-spec (#16), i18n+l10n (#15): hxlm.core.schem.HVocabHelper class drafted
4fcf3af i18n+l10n (#15): HDP will use QDP as it's own ISO 639-3 code instead of ZXX; Also, this code is not used by other ConLangs; see https://www.kreativekorp.com/clcr/,
a03af24 hdp-spec (#16), i18n+l10n (#15): HDP.quid_est_hoc_linguam() created
e48278e hdp-spec (#16), i18n+l10n (#15): fix tests; implicit namespace now don't contain .LLL.hdp.(yml|json) sufixes
3049e23 hdp-spec (#16), i18n+l10n (#15): added Russian test file
a0cb023 v0.8.1 release; hdp-spec (#16), i18n+l10n (#15): exporting to know core_vocab languages now more stable
2abc64c hdp-spec (#16), i18n+l10n (#15): HDP._get_translated_recursive() draft
b92b803 hdp-spec (#16), i18n+l10n (#15): HDP._get_translated_attr() go a level deeper... almost there...
fa9617c hdp-spec (#16), i18n+l10n (#15): country codes now have to be UPPERCASE; comment: maybe JSON Schemas could allow conditional loading, so an *.mul.hdp.yml or *.hdp.yml could allow completion all the time?; also playing with loops;
5f7193f hdp-spec (#16), i18n+l10n (#15): changed hfile to hfilum
ab12b57 i18n+l10n (#15): added core_vocab root.htransformare (old hrecipe root name)
b37218b v0.8.0 started; issues strictly related to hxlm #11, but for syntax of the YAML files, will mention the '[meta] HDP Declarative Programming (working draft) #16'