jinja[spacing] does not correctly interpret whitespace control

[alice-rc]

Summary

When you have a long jinja2 statement that you need to break up over multiple lines, it is not reading the whitespace control directives

Issue Type

• Bug Report

OS / ENVIRONMENT

ansible-lint --version
ansible-lint 6.12.1 using ansible 2.13.4

• ansible installation method: pip
• ansible-lint installation method: pip

STEPS TO REPRODUCE

- name: Set a flag if this is an vhs host
  ansible.builtin.set_fact:
    lookup_suffix: "{% if inventory_hostname | regex_search('-vhs[0-9]*$') %}_vhs
                    {%- else %}{% endif %}"

Desired Behavior

ansible-lint correctly interprets the {%- directive which removes the after _vhs

Actual Behavior

roles/myrole/tasks/pre.yml:19: jinja[spacing]: Jinja2 spacing could be improved: {% if inventory_hostname | regex_search('-vhs[0-9]*$') %}_vhs {%- else %}{% endif %} -> {% if inventory_hostname | regex_search('-vhs[0-9]*$') %}_vhs{%- else %}{% endif %} (warning)

              Rule Violation Summary
 count tag            profile rule associated tags
     1 jinja[spacing] basic   formatting (warning)

Failed after min profile: 0 failure(s), 1 warning(s) on 42 files.

[alice-rc]

It is also not picking up a # noqa jinja[spacing] on the line in question and instead wants it on the header name line. This then disables the checking for ALL the lines in that task which is NOT the desired behavior.

- name: Set version variables
      ansible.builtin.set_fact:
        cimc_desired: "{{ firmware_versions[model_lookup + fw_lookup_suffix][version_selection]['version'] }}"
        bios_desired: "{{ firmware_versions[model_lookup + fw_lookup_suffix][version_selection]
                       ['update_file_bios_version'] }}"  # noqa jinja[spacing]
        cimc_current: "{{ cisco_imc_info['active_cimc_version'] }}"
        bios_current: "{{ cisco_imc_info['imc_running_bios_version'] }}"
      vars:
        model: "{{ cisco_imc_info['cisco_model'] }}"

doesn't work, and it should. Instead the only way to suppress the check is for the entire task:

- name: Set version variables  # noqa jinja[spacing]
      ansible.builtin.set_fact:
        cimc_desired: "{{ firmware_versions[model_lookup + fw_lookup_suffix][version_selection]['version'] }}"
        bios_desired: "{{ firmware_versions[model_lookup + fw_lookup_suffix][version_selection]
                       ['update_file_bios_version'] }}"
        cimc_current: "{{ cisco_imc_info['active_cimc_version'] }}"
        bios_current: "{{ cisco_imc_info['imc_running_bios_version'] }}"
      vars:
        model: "{{ cisco_imc_info['cisco_model'] }}"

which means that it would potentially miss any jinja[spacing] issues on the other set_fact lines. In this case it was only 3 but it could be 500. Certainly not what was the desired intention.

[ssbarnea]

I converted this to a discussion because is more of support question than a clean bug report. Please use a YAML loader in order to really see how the comments work or not with multiline blocks.

I could help if you could minimize the example as much as possible, to remove anything that is not required for reproducing the issue. That process will likely make you discover that the unexpected behavior is caused by something else.

I am not saying that jinja2 spacing rule is free of bugs... ;) just that quite often what appears to be a bug proves to be behavior documented in YAML specification.

Try to use https://yaml-multiline.info/ -- Usually is a user fault if you have a quoted string "..." spreading on more than a single line.

[alice-rc]

So I don't actually understand why this is not a bug.
I have so many examples of how this is failing the check that I didn't put into the report, and it does not matter if it is a multiline string or a scalar, it just is not catching the whitespace control characters. The outcome of this is that we have to disable the entire jinja[spacing] rule as there is no way that any of our developers is going to put in noqa comments for every time this fails.
Here is maybe a better example of why this in not usable in the current form:

   - name: get_latest_swe | Set secondary date facts
      ansible.builtin.set_fact:
        __common_quarter: >-
          {% if (__common_swe_month | int) <= 3 %}1{% elif (__common_swe_month | int) <= 6 -%}2
          {%- elif (__common_swe_month | int) <= 9 %}3{% else %}4{% endif %}

roles/common/tasks/get_latest_swe.yml:1: jinja[spacing]: Jinja2 spacing could be improved: {% if (__common_swe_month | int) <= 3 %}1{% elif (__common_swe_month | int) <= 6 -%}2 {%- elif (__common_swe_month | int) <= 9 %}3{% else %}4{% endif %} -> {% if (__common_swe_month | int) <= 3 %}1{% elif (__common_swe_month | int) <= 6 -%}2{%- elif (__common_swe_month | int) <= 9 %}3{% else %}4{% endif %} (warning)

As you can see this still misses that there is NO SPACE after the '2' based on the {%- control

[NotAFile]

Encountering too. I also would consider this a bug. My case is just a vars file with:

my_variable: >-
  {% if ansible_os_family == 'Debian' -%}
  my_debian_value
  {%- else -%}
  my_other_value
  {%- endif %}

This gives:

Jinja2 spacing could be improved: {% if ansible_os_family == 'Debian' -%} my_debian_value {%- else %} my_other_value {%- endif %} -> {% if ansible_os_family == 'Debian' -%} my_debian_value{%- else %} my_other_value{%- endif %}

This makes no sense, because the space is inserted by the >- multiline string ("convert newlines to spaces"). There is no way for me to fix this. I can't even use noqa: because this is just a vars file.

[robpou]

Seeing the same in a multiline set_fact task and am in favor of this being a bug.

[nilsmeyer]

This is partially a limitation in YAML of course with there not being a way to have multiple lines that are simply joined without adding whitespaces, at least not something that isn't ugly in another way, although the {%- in Jinja should take care of any unintended spaces.

if you have something like:

my_variable: >-
  {% if ansible_os_family == 'Debian' -%}
  my_debian_value
  {%- else -%}
  my_other_value
  {%- endif %}

The "\n" after the values will be converted to a whitespace by YAML. The only way to fix it to me seems to make the code less readable and more clunky, e.g.:

my_variable: >-
  {% if ansible_os_family == 'Debian' 
  -%}my_debian_value{%- 
  else 
  -%}my_other_value{%- 
  endif %}

While this prevents unintended whitespaces it looks a lot worse than having the Jinja2 control structures on their own lines.

This is a different way to do this without tripping up ansible-lint but it requires using more python syntax instead of jinja.

{{-
if ansible_os_family == 'Debian'
'my_debian_value'
else 
'my_other_value'
-}}

I really don't think the linter should flag a space between -%} {%- as an issue.