Enhanced Form addon #349
Enhanced Form addon #349
Conversation
…XML to add Reference/Location defaulting (changes listed below)
CHANGES
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* All changed areas marked with DB$ (start/end)
* Added error handler "DisplayLogError()".
When invoked this will display errors in a popup and
show exception details in the log (gramps51.log).
* DisplayLogError() used:
* On XML SYNTAX Error in any of the forms configuration files.
This shows the file name and what the DOM reports as the issue.
* If no "<form>" tags exist in a file.
* If a "section" is missing a "role" attribute (or it's empty).
* If a "section" is missing a "type" attribute (or it's invalid).
* Some debug messages are now produced by Forms.
-d "Form Gramplet"
* The environment variable "G.FORMS" can now specify extra Form XML files that
should be loaded, these are loaded before the inbuilt ones in the order
specified.
SET G.FORMS=SameDirAsDefined.xml;xml\InAnXmlSubDirectoryFromDefined.xml
SET G.FORMS=[G.FORMS]\Australia, Queensland - Birth, Death and Marriages Register.xml;[G.FORMS]\Graves - Find-a-Grave.xml
* If there is no date or reference in the event an error message will be
displayed for you to fix that. If for some reason your form doesn't need
them then I'd suggest giving them default values date='today' and
reference='none required'.
* I display a message (and abort the save) in at least one error
situation (person unchosen) where in the past it just quietly closed.
Because of my other improvement to default a 'person" you should only
see this on 'multi' or 'family'.
* The created event how has the "Description" field filled in with
the reference followed by " @ " then the name of the source followed by
" [form]".
* For section "type=person" [others later...], you no longer have to remember
if you have selected someone or worry that you selected the wrong person
as the state is now displayed:
* Initially "no one selected yet" (in italics)
* After person creation/selection the name is displayed [small bold]
* For section "type=person" [maybe 'multi' later...], the selected person becomes
the default person (don't need to go though person selection dialog).
If multiple people selected then the first is used.
This occurs exactly once per form.
[Bug] Currently doesn't work well for multiple sections/people, all will
have same person preloaded (still better than before).
* The "title" attribute of the "<section>" tag has been improved.
If not specified (or blank) then the "role" tag's value will be used.
Also, that way, a title is always displayed, in the past you could have
completely anonymous sections on the form!!!!
* The value for the "type" attribute of a "section" is now case insensitive
('Person' == 'person').
* Fixed the Location "+" & "use existing" location button order so it is not
only consistent with the rest of the form but with Gramps in general.
* The "Headings" tab is no longer displayed if there are no headings, if it
exists then you should remember to swap to it and fill it in.
* The following applies to the existing 'date' and the new
'location' & 'reference' values (rather than creating a lot of xml attributes):
# If no "=" exists in the string then it is the "default" value, this makes
the 'date' value backwards compatible for existing forms.
# If you want a default value to contain a "=" sign you need to put
"default=" in front of it. This works since for each pair only the first
"=" is used to separate the key from the value.
# If "=" is anywhere in the value you provide then its contents are one or
more "key=value" pairs separated by "; "
(semi-colon followed by a single space, all other spaces significant).
# The order you supply keys is irrelevant.
# Incorrectly specified keys will be reported in the log (gramps51.log).
# Unspecified keys will default to some value (generally '').
* New "location" attribute on the "<form>" tag:
* Example: location="label=Where in QLD?; default=Queensland, Australia"
* [Key: default] You can now specify a default location.
If the location (place) is pre-existing then it will be used, otherwise it
will be created (and given the type "_FORM" so you can find and update it easily).
* [Key: label] You can replace the label "Location:" with your own text to better inform
the user as to the form's intention.
* New "reference" attribute on the "<form>" tag:
* Example: reference='label=QLD [B]DM #'
* [Key: label] You can replace "Reference:" with your own text to better inform the user
as to the form's intention.
* [Key: default] Can now specify a default Reference
(or perhaps a default prefix you want used).
You can replace or edit the value.
* The "date" attribute of the "<form>" tag:
* Example: date='label=Date of Birth'
* Example: date='label=Discovery Date; default=today; ro=N'
* [Key: label] You can replace "Date:" with your own text to better inform the user
as to the form's intention.
* [Key: default] The default date which can take "today" or "." to represent
todays date or any valid values like "after 1777".
* [Key: ro] The value should be "Y" (yes) or "N" (no) as to whether the date
displayed in the form is read-only or can be modified by the user.
It defaults to "Y" to be backwards compatible with existing functionality.
This is of course ignored if no "default" date was provided, as you will
need to enter it!
* Tooltips on date fields in form (for read-only & not).
* In the date entry field you can type "." as shorthand for 'today', which
you could always type, it was just longer!
If "." used the field will still be highlighted in red.
* Made the source selection buttons (NEW/EDIT) slightly larger to make
clicking them easier
TODO:
~~~~~~~~~~~~~~~~~~~~~~~
* Optional Form Help (how to fill in THIS form)
Maybe in a column on the right (HTML?)
Probably in a separate .html file (name = form id or maybe title or both)
* More validation of input. Ability to have integer, required attributes etc.
* [DO MULTI/FAMILY]
For section type=person, you no longer have to remember if you have
selected someone or worry that you selected the wrong person
as the state is now displayed:
* Initially "no one selected yet" (in italics)
* Afer person creation/selection the name is displayed [small bold]
* If # selected people = #sections then grab their names (defaulting)
How do I get the selected information (when multiple selection)?
* Prevent use OK unless all people selected - HOW TO CANCEL Button press?
* [bug] In headings tab, type in data in a field, don't leave it and press OK,
that field's data is lost.
* The "Heading" tag should be deleted and handled like all other fields,
these fields should be placed in a "heading" section and handled just
like any other exception (except that you can't choose people/family).
That way the UI is also less confusing and the user doesn't need to go
to 2 tabs to find everything. It also means all the "column"
formatting also applies to headings.
Facts / Pre-existing-Issues
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* size tag in "column" not used by Form, seems to have been obsoleted
* title tag in "form" is not used by Form, seems to have been obsoleted,
still required! Should fix this
* HeadingsTab.is_empty() doesn't work, always returns True
CAN'T WORK IT OUT (YET):
~~~~~~~~~~~~~~~~~~~~~~~~
* How to add selected people as "main participants"?
?: from ...utils.db import get_participant_from_event
MAYBE: set_participant?
SAMPLES: ~~~~~~~~~ Australia, Queensland - Birth, Death and Marriages Register.xml Graves - Find-a-Grave.xml I assume I will also need to submit these separately if the changes are accepted and update the WIKI documentation.
The environment variable "G.FORMSX" can now specify XML files to be excluded if you don't need them.
This may improve performance or be useful in testing situations. It will certainly remove some debugging output.
SET G.FORMSX=form_be.xml;form_ca.xml;form_dk.xml;form_fr.xml;form_gb.xml;form_pl.xml;form_us.xml
Now that I've tried Github, I also removed my DB$ comments as Github seems to have quite good difference handling...
Also stupid mistake, my code was based on the distributed plugin code which is NEWER than the master code so I changed fixed the differences between the two. The version was therefore updated from 2.0.24 to 2.0.20 (2.0.19 in master)
Correct Line: if str(attr.get_type()) == DEFINITION_KEY:
Yeah I believe I read another developer write here that they only update master infrequently for addons. On Travis-CI a test failed with ERROR: test_export_sql (Sqlite.tests.test_sqlite.ExportSQLTestCase), but I don't believe it is related to your PR, maybe the developers need to update how travis works? Haven't tested your update but it looks interesting especially the two new forms, can you add some screenshots what they look like? Will give it go in a few days. Only suggestions are:
Thanks for the enhancements 👍 |
|
The validation is complaining about spaces on the end of lines, doesn't say where!!!! I'm pretty sure nothing I uploaded caused that as my edit strips those when I save the files. I'm amazed that the "master" is not considered production! That made me unsure how to version the hardcoded version... About the unzip, I'm not sure how new form definition files are added (they are also currently hard-coded but a change I'm working on could remove the hard-coding). No idea what the process is (and they need to wait for my changes) that is why I zipped them, so as not to confuse people. I created the environment variables to remove the need for hardcoding while testing new forms. As for a screenshot, I've attached it, this includes form specific help (read from file) which I just added, I'm unsure if I should somehow add those changes also or what for the current "get" to complete... URL's aren't hypertext links as it seems pango doesn't support that. Help only appears if it exists OR there are headings to be filled in.
|
|
Actually I have 5 new forms: And again as I don't know how forms are expected to be handled I'm unsure what whiles they should be in, perhaps one file per source, perhaps Australian forms should go in "Australia.xml", maybe all grave related in "Grave.xml" etc, but then where would an Australian "grave" form go? Anyway I'm pretty sure I broke the code (won't work without the environment variables, now working, fixed today), you need to set the environment variables similar to the following in my batch file: |
|
Great work with the forms enhancement I really like the help idea, I assume you can collapse or dismiss the help square so it won't be annoying? Maybe the help idea can be applied to the rest of Gramps so that it has builtin help 👍 [see prculley's message for why travis-ci is not working eg: SQL addon needs to be fixed] |
|
The travis issue is because Gramps in master has some changes that are not fully compatible with the SQL addon. This would not be a problem if you were basing this on maintenance/gramps51 branch, which you should almost certainly do. I suggest that you Regarding the selection of forms and new added forms, you might want to consider changing the scan for included forms from a fixed list to a file pattern match. That way any forms that match a pattern ("*.xml" for example) would automatically get included. It would be best to name the forms according to the standard pattern "form_au.xml" for Australian forms. It is fine to add the new forms as part of the PR. BTW, @Nick-Hall originally created this addon, you might want to get his comments by asking him for a review (near top right of this web page). |
|
@wroldwiedbwe @prculley I have already made it possible to use wild cards but am unsure if loading all the forms is a good idea in any case, but I could at least handle "form_*.xml" files, I currently handle that for the environment variable, I will make that change for the forms. I see now that as I commit more files they get added to this "pull" so I know what's going on now :-) I do have an issue with grams returning wrong locale info though: I currently have my help files (zipped) in "locale.help\english" I am using GitHub on the web to do all github things, I assume this does command line magic. I'm only a beginner but I don't see why basing them on "master" is an issue, wouldn't the "maintenance" branch be the problem (not being merged in)? I assume the following is from the command line, I don't even have the files locally at this stage. I assume there would be another command to do that (can these be done with web interface). git rebase -i maintenance/gramps51 |
…ions
* Oops form wouldn't load if the environment variables didn't exist
* If a form "id" is duplicated this will be logged as a warning and the loading of that form aborted (the first occurance will be retained).
It will also tell you in which file it was originally loaded.
* If the "Select Form" would be empty a message explaining the issue and how to resolve is logged and displayed to the user.
* If headings tab exists a validation message is displayed if it was never visited.
* New optional help files get displayed if they exist.
* They contain text in pango format.
* Newlines are significant
* A new paragraph = 2 x newlines
* You can break up a long line by terminating the continuing line with
a backslash (\). All whitespace before the backslash is retained.
* Help is always added if there are headings to be entered.
* The height and width of the 'select form' dialog can be overriden by the user using the "G.FORM.SF.H" and "G.FORM.SF.W" environment variables. Defaults to original values (height too small for my liking) of 350 & 400.
* All forms env vars now begin with "G.FORM", so renamed existing:
* "G.FORMS" to "G.FORM.I"
* "G.FORMSX" to "G.FORM.X"
* The "basename" of masks listed the inclusion & exclusion
environment variables can now be a glob to match "MyForms\*.xml" etc
* The existing Form hardcoding could be removed with "form_*.xml"
As suggested filemasks are now used to load standard forms and I changed the definition list to include templates for similar form types. I now include my (currently 4) XML documents as standard (also as suggested).
# Files which may contain form definitions
definition_files = [ 'test*.xml', # Test forms have precidence (all but the first duplicate ID's are now ignored)
'custom*.xml', # User's own forms although mainly for backwards compatibility as the envvar "G.FORM.I" can be used to define your own forms.
'bdm*.xml', # Birth Death, marriages registries
'graves*.xml', # Graves/Death sites (findagrave.com, billiongraves.com and similar)
##################
'census*.xml', # Most forms are census related
'form*.xml' # Legacy naming
|
Some very general comments from me. There appear to be a lot of changes in this pull request and I strongly suggest considering whether they can be divided up into separate pull requests. Each pull request (ideally) should contain the changes which deliver a single functional enhancement or fix. Combining lots of different, independent, changes into a single pull request make it very difficult to review and therefore much less likely to be accepted. |
|
Separate pull requests would be easier to review. |
Solves this problem: On Mon, 18 May 2020 at 01:39, Paul Culley <[email protected]> wrote: I assume that you mean multiple columns in the EditForm window, 'Details' tab. The current code doesn't seem to support an easy way to have multiple columns spread over more than one row. The whole tab appears to use a Gtk Grid, with the top row as column headers and subsequent rows for the data. You would have to redesign that part of the code to utilize the grid differently. Maybe in the set_columns routines you could detect too many columns and cause the the excess to start another pair of rows. This would work for the types that are not EntryGrid, which seems to support many rows for each set of headers; I think that would need more work. Good luck Paul C. On Sat, May 16, 2020 at 4:31 PM Keith Jacobs <[email protected]> wrote: I have written a new form definition that has ten consecutive columns which I would like to split over two lines but have not managed to find out how to do it online. So I'd be grateful if someone with such knowledge could explain how to do it. -- Regards Keith Jacobs
Not sure why missed on previous upload/commit. Solves this issue: On Mon, 18 May 2020 at 01:39, Paul Culley <[email protected]> wrote: I assume that you mean multiple columns in the EditForm window, 'Details' tab. The current code doesn't seem to support an easy way to have multiple columns spread over more than one row. The whole tab appears to use a Gtk Grid, with the top row as column headers and subsequent rows for the data. You would have to redesign that part of the code to utilize the grid differently. Maybe in the set_columns routines you could detect too many columns and cause the the excess to start another pair of rows. This would work for the types that are not EntryGrid, which seems to support many rows for each set of headers; I think that would need more work. Good luck Paul C. On Sat, May 16, 2020 at 4:31 PM Keith Jacobs <[email protected]> wrote: I have written a new form definition that has ten consecutive columns which I would like to split over two lines but have not managed to find out how to do it online. So I'd be grateful if someone with such knowledge could explain how to do it. -- Regards Keith Jacobs
|
I have just added the person questions over multiple lines code, picture attached. Questions take as many lines as required and will flow if long answered entered. I will rebase my changes from "master" once I work out what needs to be done in GitHub online. I will not have time to trickle feed the changes, my wife is already complaining about how long I'm on the computer :-( I assume that would mean 1 pull for each change or do you mean commit? Either way happy to do next time, this is the first time I have used a system such as GitHub. The changes are not that big and I can list them all. It would need some testing on some of the stranger and bigger forms which I don't know about. |
…uestion per line Previous lower limit was 2 due to a weird visual issue otherwise, now fixed
That's mentioned here in the the Finding records section as "Interactive search type ahead"
Maybe maybe one of the developers can chime in but its use is limited and Yes it is part of GTK unless disabled.123 |
Yes. One pull request for each change. This may involve more than one commit. For example, code tidy up should be in a separate commit. If you have a few small related changes then it may be acceptable to put them in the same PR. |
|
A few comments:
|
* New validation: XML TITLE FIELD MUST CONTAIN A '/' FOR A FAMILY * Now using gramps locate information and renamed base help directory to "help" * I'd broken some of my error handling, now fixed again.
|
@wroldwiedbwe @Nick-Hall I will look into PEP8, is there a validation that tests for that? When Gramps installed on my Windows, it only put a few form XML files on, did it sift amongst a much larger list and only installs English ones because I'm English? By maintained by "country" did you mean "language"? I have Australian BDM forms in English, someone is going to redo those in German etc? I have "worldwide" forms written in English also. What is wrong with env vars? Settings probably better but as far as I know there aren't any for forms and I wouldn't know yet how to add them, I would move them to the settings page where possible in future. I've seen INI files being used, would that be more acceptable? The XML is not unusual, the contents of the attributes may be. As per my change, I didn't want to add a LOT of new attributes etc, if that is a show stopper I can change to defining many more attributes. I thought this would be less confusing. I'm happyish either way as I had a similar argument with myself at the time. I don't know what is expected to be done with something like "Form/formgramplet.gpr.py". Let us say I make 2 branches (and other people also do similar), I can't predict which will be accepted first so I don't know what the version number should be. Are we expected to NOT modify version numbers and that is something that the person who PUSHES the change will do? |
Yes. You will also need to split this PR.
Use
It only displays the definitions that you have linked to a source.
No. Each country has a maintainer. Translators are welcome to translate any of the definitions.
Yes. Use ini files. The form addon already uses the standard config functionality.
Create a new element. Something like the following would be more usual:
There is no need to modify the gpr file. The version number is updated as part of the build process. |
I can't see a way to create a new fork or rebaseline online, I will probably remove my fork (and recreate from the maint 51 branch as requested which I assume will kill this pull request.
Thanks but that doesn't answer the question. I don't see any form in multiple languages, am I missing something? Where is the US Census information in german, french etc?
As above I don't see those anywhere or are you saying that they are welcome but it has never been done?. I have made major improvements to the select form so I will try that, I have checked it with multiple lint tools. Hopefully, I will have more success, otherwise, I assume I can submit a new "FORM2" plugin as others seem to have done for other plugins? Or is there a 3rd party way of submitting new add-ons? The changes follow (all related to the one area as requested):
p.s. I couldn't work out how to stop the buttons being the same size and thereby making the dialog's min width wideish... And yes I could pack them but I've only started learning Python etc in the last few weeks so I take a while to work out what so-called "example" code does *when it even exists).
|
|
I did work out how to change the baseline and I have started a new pull request: #351 I can't see any way I can work on multiple things at once as pulls work on the whole repository and not selected files and you can't fork a project more than once. |
CHANGES
[WORRY] The current "master" at github contains an OLDER
version of code than the downloadable "forms gramplet"!!!!
All changes intended to be backwards compatible.
Created 2 NEW useful forms (double as sample files for syntax changes:
Added error handler "DisplayLogError()".
When invoked this will display errors in a popup and
show exception details in the log (gramps51.log).
DisplayLogError() used:
This shows the file name and what the DOM reports as the issue.
<form>tags exist in a file.Some debug messages are now produced by Forms.
-d "Form Gramplet"
The environment variable "G.FORMS" can now specify extra Form XML files that should be loaded, these are loaded before the inbuilt ones in the order specified.
SET G.FORMS=SameDirAsDefined.xml;xml\InAnXmlSubDirectoryFromDefined.xml
SET G.FORMS=[G.FORMS]\Australia, Queensland - Birth, Death and Marriages Register.xml;[G.FORMS]\Graves - Find-a-Grave.xml
The environment variable "G.FORMSX" can now specify XML files to be excluded if you don't need them.
This may improve performance or be useful in testing situations.
It will certainly remove some debugging output.
SET G.FORMSX=form_be.xml;form_ca.xml;form_dk.xml;form_fr.xml;form_gb.xml;form_pl.xml;form_us.xml
If there is no date or reference in the event an error message will be displayed for you to fix that. If for some reason your form doesn't need them then I'd suggest giving them default values date='today' and reference='none required'.
I display a message (and abort the save) in at least one error situation (person unchosen) where in the past it just quietly closed. Because of my other improvement to default a 'person" you should only see this on 'multi' or 'family'.
The created event how has the "Description" field filled in with the reference followed by " @ " then the name of the source followed by " [form]".
For section "type=person" [others later...], you no longer have to remember if you have selected someone or worry that you selected the wrong person as the state is now displayed:
* Initially "no one selected yet" (in italics)
* After person creation/selection the name is displayed [small bold]
For section "type=person" [maybe 'multi' later...], the selected person becomes the default person (you don't need to go through person selection dialog). If multiple people selected then the first is used. This occurs exactly once per-form. [Bug] Currently doesn't work well for multiple sections/people, all will have the same person preloaded (still better than before).
The "title" attribute of the
<section>tag has been improved. If not specified (or blank) then the "role" tag's value will be used. Also, that way, a title is always displayed, in the past you could havecompletely anonymous sections on the form!!!!
The value for the "type" attribute of a "section" is now case insensitive ('Person' == 'person').
Fixed the Location "+" & "use existing" location button order so it is not only consistent with the rest of the form but with Gramps in general.
The "Headings" tab is no longer displayed if there are no headings, if it exists then you should remember to swap to it and fill it in.
The following applies to the existing 'date' and the new 'location' & 'reference' values (rather than creating a lot of xml attributes):
If no "=" exists in the string then it is the "default" value, this makes
the 'date' value backwards compatible for existing forms.
If you want a default value to contain a "=" sign you need to put
"default=" in front of it. This works since for each pair only the first
"=" is used to separate the key from the value.
If "=" is anywhere in the value you provide then its contents are one or
more "key=value" pairs separated by "; "
(semi-colon followed by a single space, all other spaces significant).
The order you supply keys is irrelevant.
Incorrectly specified keys will be reported in the log (gramps51.log).
Unspecified keys will default to some value (generally '').
New "location" attribute on the
<form>tag:Example: location="label=Where in QLD?; default=Queensland, Australia"
[Key: default] You can now specify a default location. If the location (place) is pre-existing then it will be used, otherwise, it will be created (and given the type "_FORM" so you can find and update it easily).
[Key: label] You can replace the label "Location:" with your own text to better inform the user as to the form's intention.
New "reference" attribute on the
<form>tag:Example: reference='label=QLD [B]DM #'
[Key: label] You can replace "Reference:" with your own text to better inform the user
as to the form's intention.
[Key: default] Can now specify a default Reference
(or perhaps a default prefix you want used).
You can replace or edit the value.
The "date" attribute of the
<form>tag:Example: date='label=Date of Birth'
Example: date='label=Discovery Date; default=today; ro=N'
[Key: label] You can replace "Date:" with your own text to better inform the user as to the form's intention.
[Key: default] The default date which can take "today" or "." to represent today's date or any valid values like "after 1777".
[Key: ro] The value should be "Y" (yes) or "N" (no) as to whether the date displayed in the form is read-only or can be modified by the user. It defaults to "Y" to be backwards compatible with existing functionality. This is of course ignored if no "default" date was provided, as you will need to enter it!
Tooltips on date fields in form (for read-only & not).
In the date entry field you can type "." as shorthand for 'today', which you could always type, it was just longer!
If "." used the field will still be highlighted in red.
Made the source selection buttons (NEW/EDIT) slightly larger to make
clicking them easier
TODO:
Optional Form Help (how to fill in THIS form)
Maybe in a column on the right (HTML?)
Probably in a separate .html file (name = form id or maybe title or both)
More validation of input. Ability to have integer, required attributes etc.
[DO MULTI/FAMILY]
For section type=person, you no longer have to remember if you have
selected someone or worry that you selected the wrong person
as the state is now displayed:
* Initially "no one selected yet" (in italics)
* Afer person creation/selection the name is displayed [small bold]
If # selected people = #sections then grab their names (defaulting).
How do I get the selected information (when multiple selection)?
Prevent use OK unless all people selected - HOW TO CANCEL Button press?
[bug] In headings tab, type in data in a field, don't leave it and press OK,
that field's data is lost.
The "Heading" tag should be deleted and handled like all other fields, these fields should be placed in a "heading" section and handled just like any other exception (except that you can't choose people/family). That way the UI is also less confusing and the user doesn't need to go
to 2 tabs to find everything. It also means all the "column" formatting also applies to headings.
Facts / Pre-existing-Issues
still required! Should fix this
CAN'T WORK IT OUT (YET):
?: from ...utils.db import get_participant_from_event
MAYBE: set_participant?