Implement CLI hooks in netlab up/down commands

docs/customize.md

@@ -37,10 +37,11 @@ The device provisioning templates can use only built-in Jinja2 filters and a sub
 Finally, you might want to use external tools or devices not yet supported by _netlab_:
 
 * [Adding external tools](dev/extools.md) is relatively easy.
+* You can extend the functionality of **[netlab up](netlab-up)** and **[netlab down](netlab-down)** commands with [CLI hooks](dev-cli-hooks)
 * If you want to add unsupported devices to a lab but are willing to configure them manually, just [define them as _unknown_ devices](platform-unknown).
 * Adding [new functionality to an existing device](dev/device-features.md) or adding a new device to _netlab_ takes more effort. When adding a new device, it's easier to [define a new device](dev/device-box.md) and keep it _[unprovisioned](group-special-names)_ than going for a [full-blown implementation](dev/devices.md).
 
-Regardless of how far you're willing to go, we'd appreciate if you decided to [contribute your changes](dev/guidelines.md), but it's perfectly fine to keep them private. The best part: you don't have to modify the _netlab_ package to get the job done; you could use [user defaults](defaults-user-file) to define new stuff, and user-defined configuration templates (see above) to configure it.
+Regardless of how far you're willing to go, we'd appreciate it if you decided to [contribute your changes](dev/guidelines.md), but it's perfectly fine to keep them private. The best part: you don't have to modify the _netlab_ package to get the job done; you could use [user defaults](defaults-user-file) or [system defaults](defaults-locations) to define new stuff, and user-defined configuration templates (see above) to configure it.
 
 ```eval_rst
 .. toctree::

docs/dev/advanced.md

@@ -19,6 +19,7 @@ Want to know how _netlab_ works behind the scenes? These documents might give yo
 
    versioning.md
    validation.md
+   cli-hooks.md
    module-attributes.md
    vlan-interface-attributes.md
    quirks.md

docs/dev/cli-hooks.md

@@ -0,0 +1,88 @@
+(dev-cli-hooks)=
+# CLI Hooks
+
+Several [**netlab** CLI commands](netlab-cli) can execute configurable commands during their operation. This functionality is configured in **netlab._command_** [topology defaults](topo-defaults) and is currently available in **[netlab up](netlab-up)** and **[netlab down](netlab-down)** commands.
+
+Each CLI hook can specify a single CLI command (a string value). The commands executed as CLI hooks must be *executable* programs or scripts, not internal shell commands. For example, it's perfectly fine to use `touch some_file`, `bash some_script.sh`, or `python3 my_script.py`, but not `echo x`. Output redirection is not supported; if you need it, create a **bash** script to handle it.
+
+You can use the error exit status of the external commands to stop **netlab** processing -- **netlab** will display a fatal error and stop its operation whenever an external command returns a non-zero exit status.
+
+## Example
+
+You could emulate the behavior of the `netlab.lock` locking file with these CLI hooks:
+
+```
+defaults.netlab:
+  up.pre_start_lab: touch lab.lock
+  down.post_stop_lab: rm lab.lock
+```
+
+The above defaults define commands that are executed [before the lab is started](dev-cli-hooks-up) and [after the lab is stopped](dev-cli-hooks-down).
+
+After having the mechanism to create and delete the locking file, you could add **netlab.up.pre_probe** hook to check whether the locking file exists.
+
+## Environment Variables Available to CLI Hooks
+
+CLI parameters specified in the **netlab up** and **netlab down** commands are passed to the CLI hooks as environment variables starting with `NETLAB_ARGS_`. **netlab** also sets the `NETLAB_ARGS_TOPOLOGY` variable to the name of the lab topology specified in the command line or stored in the transformed lab topology snapshot file.
+
+For example:
+
+* When you execute `netlab down --cleanup`, the CLI hooks will have the `NETLAB_ARGS_CLEANUP` environment variable set to `True`. 
+* When `netlab up` is started with `-vvv` parameter, the `NETLAB_ARGS_VERBOSITY` environment variable will be set to 3.
+
+```{tip}
+* You can debug this process with the verbosity set to `-vvv` or more.
+* If you execute other **netlab** commands in the CLI hooks, those commands get the values of the `NETLAB_ARGS_` environment variables in the **defaults.args** dictionary.
+```
+
+(dev-cli-hooks-up)=
+## netlab up Hooks
+
+You can define external programs to be executed at these points in the *start the lab* process in the **defaults.netlab.up** dictionary:
+
+* **pre_probe** -- executed before the "do we have a working environment" probe. Use this command to run additional checks on your environment (for example, whether [NETLAB_MULTILAB_ID](plugin-multilab) is set).
+* **pre_start_lab** -- executed before any of the virtual machines or containers are started
+* **post_start_lab** -- executed after the virtual machines and containers are running. You can use this command to add licenses to your virtual machines.
+
+```{tip}
+*‌containerlab* does not always check whether the containers are ready. If your script needs *‌operational* network devices, execute the `netlab initial --ready` command to ensure they are ready to be configured.
+```
+
+* **pre_initial_config** and **post_initial_config** are executed before and after the initial device configuration
+* **pre_reload_config** and **post_reload_config** are executed before and after configuration reload. These commands are never executed in the same **netlab up** run as the initial device configuration hooks.
+* **pre_tools_start** and **post_tools_start** are executed before and after the external tools are started.
+
+**netlab up** can also call provider-specific hooks:
+
+* **pre_start\__provider_** before a [virtualization provider](providers) is called to start the network devices
+* **post_start\__provider_** after a [virtualization provider](providers) has started the network devices
+
+For example, you could run `netlab initial --ready` after the containerlab has started the containers to ensure the network devices are ready (Vagrant waits for the device SSH server before declaring Mission Accomplished):
+
+```
+defaults.netlab.up.post_start_clab: netlab initial --ready
+```
+
+(dev-cli-hooks-down)=
+## netlab down Hooks
+
+You can define external programs to be executed at these points in the *stop the lab* process in the **defaults.netlab.down** dictionary:
+
+* **pre_stop_lab** -- executed before the virtual machines and containers are stopped. You could use this script to deregister the licenses used by your network devices
+* **post_stop_lab** -- executed after the virtual machines and containers have been stopped. You could use this command for further licensing processing that does not require access to network devices.
+
+```{tip}
+The **‌pre_stop_lab** hook will be executed every time the **‌netlab down** command is run. The **‌post_stop_lab** hook will be executed only when the lab is successfully stopped.
+```
+
+* **pre_cleanup** -- executed before **netlab down --cleanup** starts deleting the lab configuration files.
+* **post_cleanup** -- executed after the cleanup process has been completed.
+
+```{tip}
+You can use any of the cleanup hooks to delete additional files that might have been created by your external commands.
+```
+
+**netlab down** can also call provider-specific hooks:
+
+* **pre_stop\__provider_** before a [virtualization provider](providers) is called to stop the network devices
+* **post_stop\__provider_** after a [virtualization provider](providers) has stopped the network devices

docs/netlab/config.md

@@ -70,6 +70,7 @@ router bgp {{ bgp.as }}
 
 After saving the above template into `bgp_default.j2`, you can use `netlab config bgp_default --limit somenode` to enable BGP default route advertisement and `netlab config bgp_default --limit somenode -e df_state=off` to turn it off.
 
+(netlab-config-reload)=
 ## Restoring Saved Device Configurations
 
 **netlab config --reload** implements the *reload saved device configurations* part of the **netlab initial -r** command. It waits for devices to become ready (since it's used immediately after a lab has been started) and starts the initial configuration process on devices that need more than a replay of saved configuration ([more details](netlab-up-reload)).

docs/netlab/down.md

@@ -5,6 +5,8 @@
 
 This command uses the lab topology or the snapshot file created by **netlab up** or **[netlab create](create.md)** to find the virtualization provider and executes provider-specific CLI commands to destroy the virtual lab.
 
+You can use the [CLI hooks](dev-cli-hooks) to [execute additional commands](dev-cli-hooks-down) during **netlab down** processing.
+
 ## Usage
 
 ```

docs/netlab/up.md

@@ -13,20 +13,21 @@ You can skip this step and reuse existing configuration files with the `--snapsh
 * Creates the lab management network ([more details](libvirt-mgmt))
 * Starts the virtual lab using the [selected virtualization provider](topology-reference-top-elements);
 * Performs provider-specific initialization ([more details](netlab-up-provider))
-* Deploys device configurations with **[netlab initial](initial.md)** command unless it was started with the `--no-config` flag, or reloads saved configurations if it was started with the `--reload-config` flag.
+* Deploys device configurations with **[netlab initial](initial.md)** command unless it was started with the `--no-config` flag, or [reloads saved configurations](netlab-config-reload) if it was started with the `--reload-config` flag.
 
 ![netlab up functional diagram](up.png)
 
 After configuring the lab with **netlab initial**, **netlab up** displays the [help **message** defined in the lab topology](topology-reference-top-elements).
 
+You can also use the [CLI hooks](dev-cli-hooks) to [execute additional commands](dev-cli-hooks-up) during **netlab up** processing.
+
 ```eval_rst
 .. contents:: Table of Contents
    :depth: 2
    :local:
    :backlinks: none
 ```
 
-
 ## Usage
 
 You can use `netlab up` to create configuration files and start the lab, or use `netlab up --snapshot` to start a previously created lab or restart a lab after a server reboot ([more details](netlab-up-restart)) using the transformed lab topology stored in the `netlab.snapshot.pickle` snapshot file.

docs/providers.md

@@ -1,3 +1,4 @@
+(providers)=
 # Virtualization Providers
 
 *netlab* uses third-party orchestration and virtualization tools to create, start, stop, and destroy virtual labs. It supports the following virtualization providers:

netsim/cli/__init__.py

@@ -162,6 +162,30 @@ def is_dry_run() -> bool:
   global DRY_RUN
   return DRY_RUN
 
+def set_env_args(args: argparse.Namespace, topology: typing.Optional[Box] = None) -> None:
+  """
+  Sets environment variables from CLI arguments and topology name
+
+  Primarily used by commands that call CLI hooks to give the hook scripts
+  visibility into parent command arguments (see #3355)
+  """
+  for k,v in vars(args).items():
+    if not v:                                     # Set environment variables only for
+      continue                                    # arguments specified in the command line
+    env_key = f'NETLAB_ARGS_{k.upper()}'
+    os.environ[env_key] = str(v)
+    if log.VERBOSE >= 3:                          # Tell user what we did if they asked for it
+      print(f'Setting environment variable {env_key}={v}')
+
+  if not topology:                                # Try to extract topology name
+    return                                        # Nope? Oh, well...
+  topo_input = topology.get('input',[])           # Topology name should be the first element
+  if not topo_input:                              # of the 'input' list
+    return                                        # Empty or missing list? Oh, well...
+  os.environ['NETLAB_ARGS_TOPOLOGY'] = topo_input[0]
+  if log.VERBOSE >= 3:
+    print(f'Setting NETLAB_ARGS_TOPOLOGY={topo_input[0]}')
+
 #
 # Common file/directory cleanup routine, used by collect/clab/down
 #

netsim/cli/down.py

@@ -22,6 +22,7 @@
   load_snapshot,
   parser_lab_location,
   set_dry_run,
+  set_env_args,
 )
 from .up import provider_probes
 
@@ -107,9 +108,11 @@ def stop_provider_lab(
   if sname is not None:
     exec_command = topology.defaults.providers[pname][sname].stop
 
+  external_commands.run_cli_hooks(topology.defaults,'down',f'pre_stop_{p_name}')
   p_module.call('pre_stop_lab',p_topology)
   external_commands.stop_lab(topology.defaults,p_name,"netlab down",exec_command)
   p_module.call('post_stop_lab',p_topology)
+  external_commands.run_cli_hooks(topology.defaults,'down',f'post_stop_{p_name}')
 
 '''
 lab_dir_mismatch -- check if the lab instance is running in the current directory
@@ -162,6 +165,7 @@ def stop_all(topology: Box, args: argparse.Namespace) -> None:
   providers.mark_providers(topology)
   p_module.call('pre_output_transform',topology)
 
+  external_commands.run_cli_hooks(topology.defaults,'down','pre_stop_lab')
   for s_provider in topology[p_provider].providers:
     lab_status_change(topology,f'stopping {s_provider} provider')
     try:
@@ -179,14 +183,16 @@ def stop_all(topology: Box, args: argparse.Namespace) -> None:
   except:
     if not args.force:
       sys.exit(1)
+  external_commands.run_cli_hooks(topology.defaults,'down','post_stop_lab')
 
 def run(cli_args: typing.List[str]) -> None:
   args = down_parse(cli_args)
-  set_dry_run(args)
   log.set_logging_flags(args)
+  set_dry_run(args)
 
   topology = load_snapshot(args,ghosts=False)
   mismatch = lab_dir_mismatch(topology,args)
+  set_env_args(args,topology)
 
   probes_OK = True
   external_commands.LOG_COMMANDS = True
@@ -202,12 +208,14 @@ def run(cli_args: typing.List[str]) -> None:
     stop_all(topology,args)
 
   if args.cleanup:
+    external_commands.run_cli_hooks(topology.defaults,'down','pre_cleanup')
     if 'tools' in topology:
       log.section_header('Cleanup',f'tool configuration','yellow')
       tool_cleanup(topology,True)
 
     log.section_header('Cleanup',f'configuration files','yellow')
     down_cleanup(topology,True)
+    external_commands.run_cli_hooks(topology.defaults,'down','post_cleanup')
 
   if not mismatch:
     status.remove_lab_status(topology)

netsim/cli/external_commands.py

@@ -179,13 +179,31 @@ def run_probes(settings: Box, provider: str, step: int = 0) -> None:
     log.status_success()
     print(f'{provider} installed and working correctly',flush=True)
 
-def start_lab(settings: Box, provider: str, step: int = 2, cli_command: str = "test", exec_command: typing.Optional[str] = None) -> None:
+def start_lab(
+      settings: Box,
+      provider: str,
+      step: int = 2,
+      cli_command: str = "test",
+      exec_command: typing.Optional[str] = None) -> None:
+
   if exec_command is None:
     exec_command = settings.providers[provider].start
   print_step(step,f"starting the lab -- {provider}: {exec_command}")
   if not run_command(exec_command):
     log.fatal(f"{exec_command} failed, aborting...",cli_command)
 
+def run_cli_hooks(settings: Box, cli_command: str, hook: str) -> None:
+  hook_key = f'netlab.{cli_command}.{hook}'
+  cmd = settings.get(hook_key,None)
+  if log.VERBOSE >= 2:
+    print(f"CLI hook {hook_key}: {cmd}")
+  if not cmd:
+    return
+  if log.VERBOSE:
+    log.info(f'Running {hook} CLI hook',module=cli_command,more_data=[cmd])
+  if not run_command(cmd):
+    log.fatal(f'CLI hook {hook} returned an error, aborting...',cli_command)
+
 def deploy_configs(
       command: str = "test",
       fast: typing.Optional[bool] = False,