🩹 [Patch]: Obey settings for skips (#2)

## Description

This pull request includes several changes to improve the GitHub Actions
workflows, update the documentation, and enhance the PSScriptAnalyzer
settings.

### Documentation Updates

* Updated `README.md` to improve clarity, add dependencies, and update
the example workflow section.
* Added a new `Settings.md` file to document the PSScriptAnalyzer
settings file format and key configuration options.

### Custom PSScriptAnalyzer Settings

* Added a new workflow configuration file
`.github/workflows/Action-Test-Src-Default-Custom.yml` to run tests
showing that the `Custom` settings and a `SettingsPath` works as it
should.
* Added a new custom settings file
`tests/srcTestRepo/tests/Custom.Settings.psd1` with specific rules and
configurations for PSScriptAnalyzer.
* Fixed `scripts/tests/PSScriptAnalyzer/PSScriptAnalyzer.Tests.ps1` to
include new log outputs and rule evaluation logic. This now skips tests
as per the settings file that is used.

## Type of change

<!-- Use the check-boxes [x] on the options that are relevant. -->

- [ ] 📖 [Docs]
- [ ] 🪲 [Fix]
- [ ] 🩹 [Patch]
- [ ] ⚠️ [Security fix]
- [ ] 🚀 [Feature]
- [ ] 🌟 [Breaking change]

## Checklist

<!-- Use the check-boxes [x] on the options that are relevant. -->

- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas

Author: MariusStorhaug

.github/workflows/Action-Test-Src-Default-Custom.yml

@@ -0,0 +1,24 @@
+name: Action-Test [Src-Default-Custom]
+
+run-name: "Action-Test [Src-Default-Custom] - [${{ github.event.pull_request.title }} #${{ github.event.pull_request.number }}] by @${{ github.actor }}"
+
+on:
+  workflow_dispatch:
+  pull_request:
+  schedule:
+    - cron: '0 0 * * *'
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions: {}
+
+jobs:
+  ActionTestCustom:
+    uses: ./.github/workflows/ActionTestWorkflow.yml
+    with:
+      TestType: Src-Default-Custom
+      Path: tests/srcTestRepo/src
+      Settings: Custom
+      SettingsFilePath: tests/srcTestRepo/tests/Custom.Settings.psd1

.github/workflows/Action-Test-outputs.yml

@@ -21,4 +21,3 @@ jobs:
       TestType: outputs
       Path: tests/outputTestRepo/outputs/modules/PSModuleTest
       Settings: Module
-      SettingsFilePath:

README.md

@@ -1,72 +1,66 @@
-# Invoke-ScriptAnalyzer (by PSModule)
+# Invoke-ScriptAnalyzer
 
-This repository contains a GitHub Action that runs PSScriptAnalyzer on your code.
+This repository contains a GitHub Action that runs [`PSScriptAnalyzer`](https://github.com/PowerShell/PSScriptAnalyzer) on your code.
 The action analyzes PowerShell scripts using a hashtable-based settings file to
 customize rule selection, severity filtering, and custom rule inclusion.
 
-> **Note:** This repository includes automated tests that run via Pester to ensure
-> your settings file is working as expected.
+## Dependencies
 
-## Action Details
-
-- **Name:** Invoke-ScriptAnalyzer (by PSModule)
-- **Description:** Runs PSScriptAnalyzer on the code.
-- **Author:** PSModule
-- **Branding:**
-  Icon: `check-square`
-  Color: `gray-dark`
+- This action.
+- [`PSScriptAnalyzer` module](https://github.com/PowerShell/PSScriptAnalyzer).
+- [`Invoke-Pester` action](https://github.com/PSModule/Invoke-Pester)
+- [`Pester` module](https://github.com/Pester/Pester)
+- [`GitHub-Script` action](https://github.com/PSModule/GitHub-Script)
+- [`GitHub` module](https://github.com/PSModule/GitHub)
 
 ## Inputs
 
-| Input               | Description                                                       | Required | Default                                                                     |
-|---------------------|-------------------------------------------------------------------|----------|-----------------------------------------------------------------------------|
-| **Path**            | The path to the code to test.                                     | Yes      | `${{ github.workspace }}`                                                   |
-| **Settings**        | The type of tests to run: `Module`, `SourceCode`, or `Custom`.    | No       | `Custom`                                                                    |
-| **SettingsFilePath**| If `Custom` is selected, the path to the settings file.           | No       | `${{ github.workspace }}/.github/linters/.powershell-psscriptanalyzer.psd1` |
+| Input               | Description                                                    | Required | Default                                                                     |
+|---------------------|----------------------------------------------------------------|----------|-----------------------------------------------------------------------------|
+| **Path**            | The path to the code to test.                                  | Yes      | `${{ github.workspace }}`                                                   |
+| **Settings**        | The type of tests to run: `Module`, `SourceCode`, or `Custom`. | No       | `Custom`                                                                    |
+| **SettingsFilePath**| If `Custom` is selected, the path to the settings file.        | No       | `${{ github.workspace }}/.github/linters/.powershell-psscriptanalyzer.psd1` |
 
 ## Outputs
 
-| Output  | Description                           | Value                                      |
-|---------|---------------------------------------|--------------------------------------------|
-| passed  | Indicates if the tests passed.      | `${{ steps.test.outputs.Passed }}`         |
+| Output   | Description                    | Value                              |
+|----------|--------------------------------|------------------------------------|
+| `passed` | Indicates if the tests passed. | `${{ steps.test.outputs.Passed }}` |
 
-## Files Overview
+## How It Works
 
-- **action.yml**
-  Describes the action inputs, outputs, and run steps. The action uses a
-  composite run steps approach with two main steps:
-  1. **Get test paths:** Uses a script to resolve paths and settings.
-  2. **Invoke-Pester:** Runs Pester tests against PSScriptAnalyzer.
+1. **Set a Path**
+   Choose a path for your code to test into the `Path` input. This can be a
+   directory or a file.
 
-- **scripts/main.ps1**
-  Determines the correct settings file path based on the test type. It
-  supports testing a module, source code, or using a custom settings file.
+2. **Choose settings**
+   Choose the type of tests to run by setting the `Settings` input. The options
+   are `Module`, `SourceCode`, or `Custom`. The default is `Custom`.
 
-- **scripts/tests/PSScriptAnalyzer/**
-  Contains Pester tests that run PSScriptAnalyzer using the provided settings
-  file. The tests check for issues reported by PSScriptAnalyzer based on rule
-  configuration.
+   The predefined settings:
+    - [`Module`](./scripts/tests/PSScriptAnalyzer/Module.Settings.psd1): Analyzes a module following PSModule standards.
+    - [`SourceCode`](./scripts/tests/PSScriptAnalyzer/SourceCode.Settings.psd1): Analyzes the source code following PSModule standards.
 
-## How It Works
+    You can also create a custom settings file to customize the analysis. The
+    settings file is a hashtable that defines the rules to include, exclude, or
+    customize. The settings file is in the format of a `.psd1` file.
 
-1. **Path Resolution:**
-   The action reads inputs and determines the code path, test path, and the
-   settings file path. For custom settings, it uses the file at:
-   ```powershell
-   .github/linters/.powershell-psscriptanalyzer.psd1
-   ```
-   Otherwise, it uses a default settings file from the test folder.
+    For more info on how to create a settings file, see the [Settings Documentation](./Settings.md) file.
 
-2. **Pester Testing:**
-   The tests import the settings file and use `Invoke-ScriptAnalyzer` to scan
+3. **Run the Action**
+   The tests import the settings file and use `Invoke-ScriptAnalyzer` to analyze
    the code. Each rule is evaluated, and if a rule violation is found, the test
    will fail for that rule. Rules that are marked to be skipped (via exclusions
    in the settings file) are automatically skipped in the test.
 
-3. **Automation:**
-   Designed for CI/CD, this action integrates with GitHub Actions, Azure Pipelines,
-   and other systems. The settings file customizes analysis, letting you control
-   rule inclusion, severity filtering, and custom rule paths.
+   To be clear; the action follows the settings file to determine which rules to skip.
+
+4. **View the Results**
+    The action outputs the results of the tests. If the tests pass, the action
+    will return a `passed` output with a value of `true`. If the tests fail, the
+    action will return a `passed` output with a value of `false`.
+
+    The action also outputs the results of the tests to the console.
 
 ## Example Workflow
 
@@ -81,23 +75,21 @@ jobs:
   lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - name: Checkout code
+        uses: actions/checkout@v2
+
       - name: Invoke PSScriptAnalyzer
         uses: PSModule/Invoke-ScriptAnalyzer@v1
         with:
           Path: ${{ github.workspace }}
-          Settings: Custom
-          SettingsFilePath: ${{ github.workspace }}/.github/linters/.powershell-psscriptanalyzer.psd1
+          Settings: SourceCode
 ```
 
-## Appendix: Settings File Documentation
-
-For detailed documentation on the format of the settings file, see the
-[Settings File Documentation](./SettingsFileDocumentation.md) file.
-
 ## References and Links
 
 - [PSScriptAnalyzer Documentation](https://learn.microsoft.com/powershell/module/psscriptanalyzer/)
-- [GitHub Super-Linter](https://github.com/github/super-linter)
+- [PSScriptAnalyzer Module Overview](https://learn.microsoft.com/en-us/powershell/utility-modules/psscriptanalyzer/overview?view=ps-modules)
+- [PSScriptAnalyzer Rules](https://learn.microsoft.com/en-us/powershell/utility-modules/psscriptanalyzer/rules/readme?view=ps-modules)
 - [PSScriptAnalyzer GitHub Repository](https://github.com/PowerShell/PSScriptAnalyzer)
 - [Custom Rules in PSScriptAnalyzer](https://docs.microsoft.com/powershell/scripting/developer/hosting/psscriptanalyzer-extensibility)
+- [GitHub Super-Linter](https://github.com/github/super-linter)

Settings.md

@@ -0,0 +1,83 @@
+# PSScriptAnalyzer Settings File Format Documentation
+
+This document describes the format and usage of the hashtable-based settings file
+for PSScriptAnalyzer. The file is used by the GitHub action to customize analysis.
+
+## Basic Setup
+
+The file is a PowerShell data file (.psd1) that returns a hashtable. For example:
+```powershell
+@{
+    Severity     = @('Error','Warning')
+    ExcludeRules = @('PSAvoidUsingWriteHost')
+}
+```
+This example sets the severity filter and excludes a specific rule.
+
+## Key Configuration Options
+
+- **IncludeRules**
+  A list of rules to run. Wildcards (e.g. `PSAvoid*`) are supported.
+
+- **ExcludeRules**
+  A list of rules to skip. Excludes take precedence over include lists.
+
+- **Severity**
+  Filters output by severity. Allowed values include `Error`, `Warning`, and `Information`.
+
+- **IncludeDefaultRules**
+  A Boolean switch to include default rules when using custom rules.
+
+- **CustomRulePath**
+  One or more paths to custom rule modules or scripts. These extend PSScriptAnalyzer.
+
+- **RecurseCustomRulePath**
+  Boolean to search subdirectories of the custom rule path(s) for more rule files.
+
+- **Rules**
+  A nested hashtable for rule-specific settings. Use it to pass parameters to rules.
+
+```powershell
+Rules = @{
+    PSAvoidUsingCmdletAliases = @{
+        Enabled   = $true
+        Whitelist = @('ls','gc')
+    }
+}
+```
+
+## Configuring Custom Rules
+
+Custom rules are implemented in modules (.psm1) or scripts. They must export
+functions that return DiagnosticRecord objects. Specify their location using
+`CustomRulePath`. Use `IncludeDefaultRules = $true` if you want to run both
+default and custom rules.
+
+For example:
+```powershell
+@{
+    CustomRulePath      = @('.\Modules\MyCustomRules.psm1')
+    IncludeDefaultRules = $true
+    IncludeRules        = @('PSUseApprovedVerbs', 'Measure-*')
+}
+```
+
+## Rule Execution and Skip Logic
+
+The action evaluates each rule using several configurable settings to determine whether it should be executed or skipped.
+The evaluation is performed in the following order:
+
+1. **Exclude Rules**
+   - If the rule's name is present in the **`ExcludeRules`** list, the rule is skipped immediately, regardless of other settings.
+
+2. **Include Rules**
+   - If an **`IncludeRules`** list is provided, the rule must be part of this list. If the rule's name is *not* in the list, it is skipped.
+
+3. **Severity Filtering**
+   - If a **`Severity`** list is specified, the rule's severity must be included in that list. If the rule's severity is not part of the allowed
+     values, the rule is skipped.
+
+4. **Rule-Specific Configuration**
+   - If a specific configuration exists for the rule under the **Rules** key, and its `Enable` property is set to false, the rule is skipped.
+
+To see what rules are skipped and why, check the logs for the action. There is a log group inside the test that contains the rules that were skipped.

SettingsFileDocumentation.md

@@ -38,7 +38,7 @@ runs:
         Script: ${{ github.action_path }}/scripts/main.ps1
 
     - name: Invoke-Pester
-      uses: PSModule/Invoke-Pester@fix
+      uses: PSModule/Invoke-Pester@v2
       id: test
       env:
         Settings: ${{ fromJson(steps.paths.outputs.result).Settings }}