V0.1.0

CHANGELOG.md

@@ -26,6 +26,13 @@ and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 -->
 ------
+## [0.1.0](https://github.com/asfadmin/Discovery-SearchAPI-v3/compare/v0.0.0...v0.0.1)
+
+### Added
+- Legacy API V2 Pytest suite Added, all tests passing
+
+------
+
 ## [0.0.1](https://github.com/asfadmin/Discovery-SearchAPI-v3/compare/v0.0.0...v0.0.1)
 
 ### Added

Dockerfile

@@ -1,12 +1,23 @@
-FROM public.ecr.aws/lambda/python:3.10
+FROM public.ecr.aws/docker/library/python:3.12
+COPY --from=public.ecr.aws/awsguru/aws-lambda-adapter:0.8.3 /lambda-adapter /opt/extensions/lambda-adapter
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /bin/uv
 
-# Add function code
-ADD src/SearchAPI ${LAMBDA_TASK_ROOT}
+ARG HOST=0.0.0.0
+ENV HOST=${HOST}
+ARG PORT=8080
+ENV PORT=${PORT}
+
+RUN apt update
+RUN apt clean
+RUN rm -rf /var/lib/apt/lists/*
 
 # Install the function's dependencies using file requirements.txt
-# from your project folder.
 COPY requirements.txt .
-RUN pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}" -U --no-cache-dir
+RUN uv pip install --system --no-cache-dir --upgrade -r ./requirements.txt
+
+COPY ./src/SearchAPI ./SearchAPI
+
+LABEL maintainer="Alaska Satellite Facility Discovery Team <uaf-asf-discovery@alaska.edu>"
 
 # Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
-CMD [ "main.handler" ]
+CMD ["uvicorn", "--host=$HOST", "--port=$PORT", "SearchAPI.application:app"]

README.md

@@ -1,122 +1,9 @@
 # SearchAPI-v3
 
-This project contains source code and supporting files for a serverless application that you can deploy with the SAM CLI. It includes the following files and folders.
-
-- SearchAPI - Code for the application's Lambda function.
-- events - Invocation events that you can use to invoke the function.
-- tests - Unit tests for the application code.
-- template.yaml - A template that defines the application's AWS resources.
-
-The application uses several AWS resources, including Lambda functions and an API Gateway API. These resources are defined in the `template.yaml` file in this project. You can update the template to add AWS resources through the same deployment process that updates your application code.
-
-## Default Parameters
-
-I added the parameters for deploying to `samconfig.toml` to have sensible default params specific to this project. Thus SAM may not behave the same as on other projects!
-
-## Quick start: Develop the app locally
-
-To develop against anything inside the container itself (FastAPI), run these commands to build/start the server:
-
-```bash
-sam build --template-file template-docker.yaml
-sam local start-api --env-vars local-env-vars.conf
-```
-
-If you haven't done so, you'll have to pull the `asf_search` repo to the root of this project (and switch to the `cs.searchapi-v3-edits` branch if it's not merged yet).
-
-Once that branch is merged, we can remove the local install from the dockerfile and delete the local repo. This helps with developing against asf_search until then.
-
-## Deploy the sample application
-
-To use the SAM CLI, you need the following tools.
-
-- SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
-- [Python 3 installed](https://www.python.org/downloads/)
-- Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community)
-
-To build and deploy your application for the first time, run the following in your shell:
-
-### For Python-based Lambda
-
-```bash
-export AWS_PROFILE=<your-profile>
-sam validate --template-file template-python.yaml
-sam build --template-file template-python.yaml
-sam package
-sam deploy /
-    --stack-name SearchAPI-v3-SAM-python
-```
-
-### For Docker-based Lambda
-
-```bash
-export AWS_PROFILE=<your-profile>
-sam validate --template-file template-docker.yaml
-sam build --template-file template-docker.yaml
-sam package --image-repository "$(aws sts get-caller-identity --query Account --output text).dkr.ecr.us-east-1.amazonaws.com/searchapi-v3"
-sam deploy \ 
-    --stack-name SearchAPI-v3-SAM-docker \ 
-    --image-repository "$(aws sts get-caller-identity --query Account --output text).dkr.ecr.us-east-1.amazonaws.com/searchapi-v3"
-```
-
-## Use the SAM CLI to build and test locally
-
-Build your application with the `sam build ...` command above.
-
-The SAM CLI installs dependencies defined in `SearchAPI/requirements.txt`, creates a deployment package, and saves it in the `.aws-sam/build` folder.
-
-Test a single function by invoking it directly with a test event. An event is a JSON document that represents the input that the function receives from the event source. Test events are included in the `events` folder in this project.
-
-Run functions locally and invoke them with the `sam local invoke` command.
-
-```bash
-# THIS WON'T WORK until we add API Gateway events to events/*
-# for now, use `sam local start-api` in the next part instead.
-sam local invoke SearchApiFunction --event events/event.json
-```
-
-The SAM CLI can also emulate your application's API. Use the `sam local start-api` to run the API locally on port 3000.
-
-```bash
-sam local start-api --env-vars local-env-vars.conf
-curl http://localhost:3000/
-```
-
-## Fetch, tail, and filter Lambda function logs
-
-To simplify troubleshooting, SAM CLI has a command called `sam logs`. `sam logs` lets you fetch logs generated by your deployed Lambda function from the command line. In addition to printing the logs on the terminal, this command has several nifty features to help you quickly find the bug.
-
-`NOTE`: This command works for all AWS Lambda functions; not just the ones you deploy using SAM.
-
-```bash
-sam logs -n SearchApiFunction --stack-name SearchAPI-v3-SAM --tail
-```
-
-You can find more information and examples about filtering Lambda function logs in the [SAM CLI Documentation](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-logging.html).
-
-## Tests
-
-Tests are defined in the `tests` folder in this project. Use PIP to install the test dependencies and run tests.
-
-```bash
-pip install -r tests/requirements.txt --user
-# unit test
-python -m pytest tests/unit -v
-# integration test, requiring deploying the stack first.
-# Create the env variable AWS_SAM_STACK_NAME with the name of the stack we are testing
-AWS_SAM_STACK_NAME=<stack-name> python -m pytest tests/integration -v
-```
-
-## Cleanup
-
-To delete the sample application that you created, use the AWS CLI. Assuming you used your project name for the stack name, you can run the following:
-
-```bash
-sam delete --stack-name SearchAPI-v3-SAM
-```
-
-## Resources
-
-See the [AWS SAM developer guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html) for an introduction to SAM specification, the SAM CLI, and serverless application concepts.
-
-Next, you can use AWS Serverless Application Repository to deploy ready to use Apps that go beyond hello world samples and learn how authors developed their applications: [AWS Serverless Application Repository main page](https://aws.amazon.com/serverless/serverlessrepo/)
+- Login to aws console using Kion
+- Find the region that has the VPC and note account number, vpc_id, subnet_ids and security_group. Subnet_ids should be a comma separated list
+- Get temp cli credentials from Kion and add them to your shell
+- Run CDK bootstrap in region with the VPC using cdk-bootstrap-example.sh and filling in the account number, vpc_id, subnet_ids and security_group
+- If not already created, make an GitHubActionsOidcProvider using the cdk/oidc/oidc-provider.yml template
+- Create OIDC role using cloudformation template cdk/oidc/github-actions-oidc.yml. For 'ActionsRoleName' parameter put 'SearchAPIActionsOIDCRole'
+- Create a github environment with params using the same values from the CDK bootstrap. AWS_ACCOUNT_ID, SECURITY_GROUP, SUBNET_IDS, VPC_ID

cdk/cdk/cdk_stack.py

@@ -1,5 +1,6 @@
 from aws_cdk import (
     Stack,
+    Duration,
     aws_lambda as lambda_,
     aws_apigateway as apigateway,
     aws_ec2 as ec2
@@ -33,6 +34,7 @@ def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
         search_api_lambda = lambda_.DockerImageFunction(
             self,
             "SearchAPIFunction",
+            timeout=Duration.seconds(30),
             code=lambda_.DockerImageCode.from_image_asset(
                 directory='..'
                 ),
@@ -43,7 +45,7 @@ def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
         )
 
         api = apigateway.LambdaRestApi(
-            self,
+            self,   
             "search-api-gateway",
             handler=search_api_lambda,
             endpoint_configuration=apigateway.EndpointConfiguration(

cdk/oidc/oidc-provider.yml

@@ -0,0 +1,12 @@
+AWSTemplateFormatVersion: 2010-09-09
+Description: Create GitHub OIDC provider
+
+Resources:
+  GitHubActionsOidcProvider:
+    Type: AWS::IAM::OIDCProvider
+    Properties:
+      ClientIdList:
+        - sts.amazonaws.com
+      ThumbprintList:
+        - 6938fd4d98bab03faadb97b34396831e3780aea1
+      Url: https://token.actions.githubusercontent.com

cdk/tests/unit/test_cdk_stack.py

@@ -1,13 +1,13 @@
 import aws_cdk as core
 import aws_cdk.assertions as assertions
 
-from cdk.cdk_stack import CdkStack
+from cdk.cdk_stack import SearchAPIStack
 
 # example tests. To run these tests, uncomment this file along with the example
 # resource in cdk/cdk_stack.py
 def test_sqs_queue_created():
     app = core.App()
-    stack = CdkStack(app, "cdk")
+    stack = SearchAPIStack(app, "cdk")
     template = assertions.Template.from_stack(stack)
 
 #     template.has_resource_properties("AWS::SQS::Queue", {

conftest.py

@@ -0,0 +1,68 @@
+import argparse
+import requests
+import warnings
+import datetime
+import time
+
+from SearchAPI.application.asf_env import load_config_file
+
+def api_type(user_input: str) -> str:
+    # If it's a url with a trailing '/', remove it:
+    if user_input.endswith('/'):
+        user_input = user_input[:-1]
+    # Grab list of maturities, for available API's:
+    maturities = load_config_file()
+    api_info = None # Will be: ("api url: str", "is_flex_maturity: bool")
+    assert user_input.lower() != "default", "This is used when the URL isn't on this list. Don't call directly! Just make `url <Your actuall url>` to use."
+    for nickname, info in maturities.items():
+        if nickname.lower() == "default":
+            continue
+        # If the url in maturities ends with '/', remove it. (Lets it match user input always):
+        if info["this_api"].endswith('/'):
+            info["this_api"] = info["this_api"][:-1]
+        # If you gave it the nickname, or the url of a known api:
+        if user_input.lower() in [ nickname.lower(), info["this_api"].lower(), ]:
+            api_info = {
+                "this_api": info["this_api"],
+                "flexible_maturity": info["flexible_maturity"],
+            }
+            break
+    # If you don't hit an option in maturities.yml, assume what was passed IS the url:
+    if api_info is None:
+        warnings.warn(f"API url not found in 'maturities.yml'. Using it directly. ({user_input}).")
+
+        api_info = maturities["default"]
+        api_info["this_api"] = user_input
+
+    # Assume it's a url now, and try to connect:
+    # Try for a bit. It's possible lambda isn't up yet or something
+    endTime = datetime.datetime.now() + datetime.timedelta(minutes=2)
+    while datetime.datetime.now() < endTime:
+        try:
+            r = requests.get(api_info["this_api"], timeout=30)
+        except (requests.exceptions.Timeout, requests.exceptions.ConnectionError):
+            # If it throws instantly, don't bombard the API:
+            time.sleep(2.0)
+            # Jump back up to the top and try again:
+            continue
+        if r.status_code == 200:
+            # It connected!! You're good:
+            return api_info
+    raise argparse.ArgumentTypeError(f"ERROR: Could not connect to url '{user_input}'.")
+
+def string_to_bool(user_input: str) -> bool:
+    user_input = str(user_input).upper()
+    if 'TRUE'.startswith(user_input):
+       return True
+    elif 'FALSE'.startswith(user_input):
+       return False
+    else:
+       raise argparse.ArgumentTypeError(f"ERROR: Could not convert '{user_input}' to bool (true/false/t/f).")
+
+def pytest_addoption(parser):
+    parser.addoption("--api", action="store", type=api_type, default="local",
+        help = "Which API to hit when running tests (LOCAL/DEV/TEST/PROD, or url)."
+    )
+    parser.addoption("--flex", action="store", type=string_to_bool,
+        help = "'flexible_maturity': wether to attach 'maturity' to the URL strings."
+    )

requirements.txt

@@ -2,33 +2,27 @@ anyio==3.6.2
 certifi>=2023.7.22
 click==8.1.3
 dnspython==2.3.0
-email-validator==1.3.1
-fastapi>=0.109.1
+email-validator>=2.0
+fastapi>=0.115.12
 h11==0.14.0
 httpcore==0.16.3
-httptools==0.5.0
 httpx==0.23.3
 idna==3.4
 itsdangerous==2.1.2
 jinja2>=3.1.3
-MarkupSafe==2.1.2
 orjson>=3.9.15
-pydantic==2.4.2
+pydantic==2.8.2
 python-dotenv==1.0.0
 python-multipart>=0.0.7
-PyYAML==6.0
+PyYAML==6.0.2
 rfc3986==1.5.0
 sniffio==1.3.0
-starlette>=0.36.2
 typing_extensions==4.10.0
 ujson==5.7.0
 uvicorn==0.21.1
-uvloop==0.17.0
 watchfiles==0.19.0
-websockets==10.4
-mangum
 
-asf_search==8.0.0
+asf_search==8.1.0
 python-json-logger==2.0.7
 
 pyshp