Tests did not run sucessfully with phpIPAM v1.7.x

[cmeissner]

Describe the bug

Running the complete test suite of this repository against a phpIPAM version 1.7.x will fail for several modules.

====== short test summary info ======
FAILED tests/test_crud.py::test_crud[domain_vlan_mapping]
FAILED tests/test_crud.py::test_crud[domain_vlan_subnet_mapping]
FAILED tests/test_crud.py::test_crud[domain]
FAILED tests/test_crud.py::test_crud[device]
FAILED tests/test_crud.py::test_crud[nameserver]
FAILED tests/test_crud.py::test_crud[subnet]
FAILED tests/test_crud.py::test_crud[subnet_vrf_mapping]
FAILED tests/test_crud.py::test_crud[vrf]
====== 8 failed, 10 passed in 19.51s ======

To Reproduce
Steps to reproduce the behavior:

1. running make clean
2. set PHPIPAM_VERSION to 1.7.0
3. running make setup-phpipam
4. running make test-example_setup
5. running make test-all

Observe the command output.

Expected behavior

All tests should run without failures, with the following output:

========== 18 passed in 22.33s ==========

Versions:

• Ansible:

  $ ansible [core 2.15.12]
  config file = /home/cme/Development/github/cmeissner/phpipam-ansible-modules/ansible.cfg
  configured module search path = ['/home/cme/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/cme/.pyenv/versions/3.9.11/envs/pam_3.9.11/lib/python3.9/site-packages/ansible
  ansible collection location = /home/cme/Development/github/cmeissner/phpipam-ansible-modules/build/collections
  executable location = /home/cme/.pyenv/versions/3.9.11/envs/pam_3.9.11/bin/ansible
  python version = 3.9.11 (main, Mar 21 2022, 19:15:19) [GCC 11.2.1 20220127 (Red Hat 11.2.1-9)] (/home/cme/.pyenv/versions/3.9.11/envs/pam_3.9.11/bin/python)
  jinja version = 3.1.4
  libyaml = True

• phpipam-ansible-modules:

  $ ansible-galaxy collection list codeaffen.phpipam
  
  # /home/cme/Development/github/cmeissner/phpipam-ansible-modules/build/collections/ansible_collections
  Collection        Version
  ----------------- -------
  codeaffen.phpipam 1.7.0 

• phpypam:

  $ pip freeze | grep phpypam
  phpypam==1.0.2

• phpIPAM:

  $ Version from webUI
  v1.7.0

• python version

  $ python -V
  Python 3.9.11

Additional context

This is a meta bug. It should be used to investigate all affected modules and create subsequent bug reports to fix each bug separately.

[cmeissner]

There are breaking changes in API responses from phpIPAM version 1.7. I illustrate the difference by showing the results of the following command against 1.6.0 and 1.7.0:

curl -sk -XGET -H "phpipam-token: ${PHPIPAM_TOKEN}" "https://localhost:8443/api/ansible/subnets/1" | jq

phpIPAM <= 1.6

There were no real integer values, everything was strings. Also, no bools, these values were also strings. See the returned values for a subnet:

{                                                                                                                                                                                                                                                
  "code": 200,                                                                                                                                                                                                                                   
  "success": true,                                                                                                                                                                                                                               
  "data": {                                                                                                                                                                                                                                      
    "id": "1",                                                                                                                                                                                                                                   
    "subnet": "fd13:6d20:29dc:cf27::",                                                                                                                                                                                                           
    "mask": "64",                                                                                                                                                                                                                                
    "sectionId": "2",                                                                                                                                                                                                                            
    "description": "Private subnet 1",                                                                                                                                                                                                           
    "linked_subnet": null,                                                                                                                                                                                                                       
    "firewallAddressObject": null,                                                                                                                                                                                                               
    "vrfId": "0",                                                                                                                                                                                                                                
    "masterSubnetId": "0",                                                                                                                                                                                                                       
    "allowRequests": "1",                                                                                                                                                                                                                        
    "vlanId": "1",                                                                                                                                                                                                                               
    "showName": "1",                                                                                                                                                                                                                             
    "device": "0",                                                                                                                                                                                                                               
    "permissions": "{\"3\":\"1\",\"2\":\"2\"}",                                                                                                                                                                                                  
    "pingSubnet": "0",                                                                                                                                                                                                                           
    "discoverSubnet": "0",                                                                                                                                                                                                                       
    "resolveDNS": "0",                                                                                                                                                                                                                           
    "DNSrecursive": "0",                                                                                                                                                                                                                         
    "DNSrecords": "0",                                                                                                                                                                                                                           
    "nameserverId": "0",                                                                                                                                                                                                                         
    "scanAgent": null,                                                                                                                                                                                                                           
    "customer_id": null,                                                                                                                                                                                                                         
    "isFolder": "0",                                                                                                                                                                                                                             
    "isFull": "0",                                                                                                                                                                                                                               
    "isPool": "0",                                                                                                                                                                                                                               
    "tag": "2",                                                                                                                                                                                                                                  
    "threshold": "0",                                                                                                                                                                                                                            
    "location": null,                                                                                                                                                                                                                            
    "editDate": null,                                                                                                                                                                                                                            
    "lastScan": null,
    "lastDiscovery": null,
    "calculation": {
      "Type": "IPv6",
      "Host address": "/",
      "Host address (uncompressed)": "/",
      "Subnet prefix": "fd13:6d20:29dc:cf27::/64",
      "Prefix length": "64",
      "Subnet Reverse DNS": "7.2.f.c.c.d.9.2.0.2.d.6.3.1.d.f.ip6.arpa",
      "Min host IP": "fd13:6d20:29dc:cf27::",
      "Max host IP": "fd13:6d20:29dc:cf27:ffff:ffff:ffff:ffff",
      "Number of hosts": "18446744073709551616",
      "Address type": "NET_IPV6"
    }
  },
  "time": 0.004
}

phpIPAM >= 1.7

{
  "code": 200,
  "success": true,
  "data": {
    "id": 1,
    "subnet": "fd13:6d20:29dc:cf27::",
    "mask": "64",
    "sectionId": 2,
    "description": "Private subnet 1",
    "linked_subnet": null,
    "firewallAddressObject": null,
    "vrfId": 0,
    "masterSubnetId": 0,
    "allowRequests": 1,
    "vlanId": 1,
    "showName": 1,
    "device": 0,
    "permissions": "{\"3\":\"1\",\"2\":\"2\"}",
    "pingSubnet": 0,
    "discoverSubnet": 0,
    "resolveDNS": 0,
    "DNSrecursive": 0,
    "DNSrecords": 0,
    "nameserverId": 0,
    "scanAgent": null,
    "customer_id": null,
    "isFolder": 0,
    "isFull": 0,
    "isPool": 0,
    "tag": 2,
    "threshold": 0,
    "location": null,
    "editDate": null,
    "lastScan": null,
    "lastDiscovery": null,
    "calculation": {
      "Type": "IPv6",
      "Host address": "/",
      "Host address (uncompressed)": "/",
      "Subnet prefix": "fd13:6d20:29dc:cf27::/64",
      "Prefix length": "64",
      "Subnet Reverse DNS": "7.2.f.c.c.d.9.2.0.2.d.6.3.1.d.f.ip6.arpa",
      "Min host IP": "fd13:6d20:29dc:cf27::",
      "Max host IP": "fd13:6d20:29dc:cf27:ffff:ffff:ffff:ffff",
      "Number of hosts": "18446744073709551616",
      "Address type": "NET_IPV6"
    }
  },
  "time": 0.034
}

Conclusion

With these findings, we need to make a greater refactoring to change the behavior of handling Integers and Booleans for the different versions of phpIPAM.

[cmeissner]

In phpIPAM - 1.7 changelog, I found that:

Added $api_stringify_results config.php option for php81 API backwards compatibility;

Further investigations in config.dist.php show the following comment:

/**
 * PHP8.1 - Integers and floats in result sets will now be returned using native PHP types instead of strings when using emulated prepared statements.
 * Add option to restore prior behaviour for API consumers.
 *                       
 * Can be overwritten via "api-stringify-results: 0|1" header in API requests.
 */                                            
                                                             
$api_stringify_results = false;

So a feasible solution for supporting phpIPAM 1.7 and above we can either document that 1.7 users need to set $api_stringify_results = true; as a first workaround.

To solve the situation on our end, we could implement a configuration parameter or an environment variable to set the used version to 1.7 and provide the header with all API interactions. To implement the final solution, codeaffen/phpypam#57 needs to be fixed first.

[cmeissner]

Issue closed, as a configuration change on phpIPAM side will solve the issue. A later enhancement could be implemented if needed.