[BUG] with empy wanted_pkgs and 2019.2 fails on pkg.installed

[arthurzenika]

Your setup

Formula commit hash / release tag

7d63859

Versions reports (master & minion)

salt-call --versions-report
Salt Version:
           Salt: 2019.2.2
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Sep 26 2018, 18:42:22)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.11 
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-11-amd64
         system: Linux
        version: debian 9.11 

Pillar / config used

packages:
  pkgs:
    wanted:

---

Bug details

Describe the bug

----------
          ID: wanted_pkgs
    Function: pkg.installed
      Result: False
     Comment: Attempt 1: Returned a result of "False", with the following comment: "Problem encountered installing package(s). Additional info follows:
              
              errors:
                  - Running scope as unit: run-r277f85bc77b94f668eefedd7cedd7a54.scope
                    E: Unable to locate package wanted_pkgs"
              Attempt 2: Returned a result of "False", with the following comment: "Problem encountered installing package(s). Additional info follows:
              
              errors:
                  - Running scope as unit: run-red3c26eba2cf440487d115a396506e48.scope
                    E: Unable to locate package wanted_pkgs"
              Problem encountered installing package(s). Additional info follows:
              
              errors:
                  - Running scope as unit: run-rec8c9a5c750d4ce9882b5ec09b725ccc.scope
                    E: Unable to locate package wanted_pkgs
     Started: 13:17:32.463431
    Duration: 131452.439 ms
     Changes:   

Steps to reproduce the bug

Expected behaviour

Attempts to fix the bug

Additional context

[arthurzenika]

pillar.get shows :

packages:
  pkgs:
    wanted:
      None

[myii]

@arthurlogilab Since wanted is a list, you'll need to provide an empty list:

packages:
  pkgs:
    wanted: []

• I've confirmed that works locally.

The pillar.example could be updated to show this.

[myii]

@arthurlogilab Taking this further, the default empty list is already set:

packages-formula/packages/defaults.yaml

Line 9 in 7d63859

wanted: []

So ignoring wanted completely still works:

packages:
  pkgs:
    held:
      alien: 8.95
      iotop: 0.6-2
    unwanted:
      - avahi-daemon
    required:
      pkgs:
        - wget
        - git

Can we close this issue or is there something else you believe needs to be done?

[arthurzenika]

In some cases when generating pillars with complex jinja it can be difficult to get the [] you suggest, that's why I'd be in favor of handling this None case in the formula.

[javierbertoli]

@arthurlogilab , as @myii mentioned, defaults.yaml has all the variables set to reasonable values so the formula can work. In the case of an empty list for this parameter, you can just skip setting it, and the formula will fill an empty list to it.

Your situation, if I understand you correctly, it that you're generating a bunch of pillars and the generator's logic is setting packages.pkgs.wanted even when there are no packages to add and, instead of setting it to an empty list is setting it to None.

On one hand, I could argue that the issue is in the generator and that the issue with the formula, at most, is that it does not clearly documents that these vars should be list and set to an empty list or left unset when empty.

OTOH, I'm not against adding None as a special parameter to manage as you suggest, but I have a few thoughts about that case, that I'd like to discuss:

Saltstack or pillars managers (pillarstack, etc) many times returns None when the pillar variable is incorrectly set. In that case, that's an error that should be raised.

If we begin taking None as a synonym to an empty list, we'll be masking this error and passing OK when we shouldn't.

I can't think of a way to differentiate the intentional None (such as in your case) with the errored None, so we can add it but also signal errors where appropriate.

Do you have any suggestions?