Skip to content

Abort Censys annotation if we run out of credits #81

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
phillip-stephens wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Abort Censys annotation if we run out of credits #81

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2c021a0
error out if we hit a censys error that we can't recover from
phillip-stephens May 7, 2026
32f1e8e
document new behavior of quitting on api credit exhaustion
phillip-stephens May 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,8 @@ They offer a free tier that allows for a limited number of queries per month, wh

> [!NOTE]
> The free account (as of April 2026) allows a single concurrent request and 100 requests per month.
> Once you've used all your credits, the `censys:` annotation will be empty for any following IPs until your credits refill the following month.
> Once you've used all your credits, the Censys API will return an error upon further requests until your credits reset at the beginning of the next month or you purchase additional credits.
> ZAnnotate will quit with an error if this occurs to prevent silent errors, so if you have a large dataset to annotate, be mindful of your credit usage and prioritize only annotating the IPs you have credits for.
> With the free account only offering a single concurrent request, you'll want to leave `--censys-threads=1` unless you pay for a higher tier.

1. Create an account at [Censys.io](https://censys.io) and get a Personal Access Token (PAT) from Personal Settings > Personal Access Tokens.
Expand Down
45 changes: 19 additions & 26 deletions censys.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"io"
"net"
"net/http"
"strings"

log "github.com/sirupsen/logrus"
)
Expand Down Expand Up @@ -103,39 +104,31 @@ func (a *CensysAnnotator) Annotate(ip net.IP) interface{} {
log.Debugf("failed to close response body: %v", err)
}
}(res.Body)
if res.StatusCode != http.StatusOK {
log.Debugf("failed to annotate ip %s with censys: %s", ip.String(), res.Status)
body, _ := io.ReadAll(res.Body)
if res.StatusCode >= 400 && res.StatusCode < 500 {
// From https://docs.censys.com/reference/get-started#step-6-handle-http-response-codes
// 4XX errors are not transient and so we'll abort and report to the user
log.Fatalf("censys api returned an http status '%s' with message: '%s'. "+
"Cannot continue to annotate, please check that you have sufficient API credits and your PAT is correct", res.Status, strings.TrimSpace(string(body)))

} else if res.StatusCode != http.StatusOK {
// Should be a transient error, log and move on
log.Debugf("censys api returned an http %s status with message: '%s'. Skipping Censys annotation for this IP: %s", res.Status, strings.TrimSpace(string(body)), ip.String())
return nil
}
body, _ := io.ReadAll(res.Body)
var result any
// We have a successful response, unmarshall it.
// Struct taken from v1.1 of Censys API docs
var result struct {
Result struct {
Resource any `json:"resource"`
} `json:"result"`
}
err = json.Unmarshal(body, &result)
if err != nil {
log.Debugf("failed to parse censys response for ip %s: %v", ip.String(), err)
return nil
}
// By default, Censys' result is wrapped in a result[resource[real_data]]. We'll unwrap that here
dropResultCast, ok := result.(map[string]interface{})
if !ok {
log.Debugf("failed to unwrap censys response for ip %s: %v", ip.String(), result)
return result
}
dropResult := dropResultCast["result"]
if dropResult == nil {
log.Debugf("failed to unwrap censys response for ip %s: %v", ip.String(), result)
return result
}
dropResourceCast, ok := dropResult.(map[string]interface{})
if !ok {
log.Debugf("failed to unwrap censys response for ip %s: %v", ip.String(), result)
return result
}
dropResource := dropResourceCast["resource"]
if dropResource == nil {
log.Debugf("failed to unwrap censys response for ip %s: %v", ip.String(), result)
return result
}
return dropResource
return result.Result.Resource
}

func (a *CensysAnnotator) Close() error {
Expand Down
Loading