Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions go/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,9 @@ ql/lib/go.dbscheme.stats: ql/lib/go.dbscheme build/stats/src.stamp extractor
codeql dataset measure -o $@ build/stats/database/db-go

test: all build/testdb/check-upgrade-path
codeql test run -j0 ql/test --search-path .. --check-diff-informed --consistency-queries ql/test/consistency --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo) --check-databases --fail-on-trap-errors --check-undefined-labels --check-unused-labels --check-repeated-labels --check-redefined-labels --check-use-before-definition
codeql test run -j0 ql/test --search-path .. --check-diff-informed --consistency-queries ql/consistency-queries --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo) --check-databases --fail-on-trap-errors --check-undefined-labels --check-unused-labels --check-repeated-labels --check-redefined-labels --check-use-before-definition
# use GOOS=linux because GOOS=darwin GOARCH=386 is no longer supported
env GOOS=linux GOARCH=386 codeql$(EXE) test run -j0 ql/test/query-tests/Security/CWE-681 --search-path .. --check-diff-informed --consistency-queries ql/test/consistency --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo)
env GOOS=linux GOARCH=386 codeql$(EXE) test run -j0 ql/test/query-tests/Security/CWE-681 --search-path .. --check-diff-informed --consistency-queries ql/consistency-queries --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo)
cd extractor; $(BAZEL) test ...
bash extractor-smoke-test/test.sh || (echo "Extractor smoke test FAILED"; exit 1)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
category: breaking
---
* The query `go/unexpected-frontend-error` has been moved from the `codeql/go-queries` query to the `codeql-go-consistency-queries` query pack.
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,25 @@

private import go
private import DataFlowImplSpecific as Impl
private import DataFlowUtil
private import TaintTrackingImplSpecific
private import codeql.dataflow.internal.DataFlowImplConsistency
private import semmle.go.dataflow.internal.DataFlowNodes

private module Input implements InputSig<Location, Impl::GoDataFlow> { }
private module Input implements InputSig<Location, Impl::GoDataFlow> {
predicate missingLocationExclude(DataFlow::Node n) {
n instanceof DataFlow::GlobalFunctionNode or n instanceof Private::FlowSummaryNode
}

predicate uniqueNodeLocationExclude(DataFlow::Node n) { missingLocationExclude(n) }

predicate localFlowIsLocalExclude(DataFlow::Node n1, DataFlow::Node n2) {
n1 instanceof DataFlow::FunctionNode and simpleLocalFlowStep(n1, n2, _)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still like to understand what these steps are for, and why the cannot be modeled as jump steps?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you see my comment here? It's so we can track which functions are called, even when calling via a variable. Local flow is so that when we look for callees we can just use local flow (this may miss some results, but in practice it seems good enough). I don't know how other languages where functions can easily be assigned to variables deal with this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see why you suggest jump steps, as it is a lot like reading a global variable. I guess the difference is that a global function node is a like global variable that has a fixed value that can't ever change, so we don't have to worry about the call context.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you see my comment here? It's so we can track which functions are called, even when calling via a variable. Local flow is so that when we look for callees we can just use local flow (this may miss some results, but in practice it seems good enough). I don't know how other languages where functions can easily be assigned to variables deal with this.

Sorry, I missed that.

This also happens in Rust and C#, but there we make use of the built-in support for resolving closures in the data flow library. See e.g.

// A path expression, that resolves to a function, evaluates to a function
// pointer. Except if the path occurs directly in a call, then it's just a
// call to the function and not a function being passed as data.
resolvePath(e.(PathExpr).getPath()) = c.asCfgScope() and
not any(CallExpr call).getFunction() = e
.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just occurred to me that this will have to be changed to get overlay databases to work, as we will want local flow to be local. When that happens I will try to remember to remove this exclusion from the consistency checks.

}

predicate argHasPostUpdateExclude(DataFlow::ArgumentNode n) {
not DataFlow::insnHasPostUpdateNode(n.asInstruction())
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you forget to add the DataFlowConsistency.ql file?

Copy link
Contributor Author

@owen-mc owen-mc Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It already exists, so you won't see it in this PR. I made it a while ago but didn't set up CI to run it because there were so many results due to not using use-use flow at the time.


module Consistency = MakeConsistency<Location, Impl::GoDataFlow, GoTaintTracking, Input>;
1 change: 0 additions & 1 deletion go/ql/test/consistency/UnexpectedFrontendErrors.expected

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
identityLocalStep
| main.go:46:18:46:18 | n | Node steps to itself |
| main.go:47:3:47:3 | c | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
reverseRead
| timing.go:15:18:15:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| timing.go:28:18:28:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| timing.go:41:18:41:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| timing.go:53:18:53:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
reverseRead
| ImproperLdapAuth.go:18:18:18:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| ImproperLdapAuth.go:39:18:39:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| ImproperLdapAuth.go:64:18:64:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
reverseRead
| go-jose.v3.go:16:17:16:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| golang-jwt-v5.go:22:17:22:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
reverseRead
| DivideByZero.go:10:12:10:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:17:12:17:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:24:12:24:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:31:12:31:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:38:12:38:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:45:12:45:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:54:12:54:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:63:12:63:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:72:12:72:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
identityLocalStep
| DatabaseCallInLoop.go:9:3:9:4 | db | Node steps to itself |
| test.go:21:12:21:13 | db | Node steps to itself |
| test.go:25:15:25:16 | db | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
reverseRead
| test.go:60:15:60:21 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:61:24:61:30 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:62:13:62:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:63:17:63:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:64:8:64:14 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:65:12:65:18 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:66:8:66:14 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:67:12:67:18 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:68:17:68:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:69:21:69:27 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:70:13:70:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:71:17:71:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:72:16:72:22 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:73:20:73:26 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:74:7:74:13 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:75:11:75:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:76:9:76:15 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:77:13:77:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:78:18:78:24 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:79:22:79:28 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:80:5:80:11 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:81:9:81:15 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:82:7:82:13 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:83:11:83:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:84:15:84:21 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:85:16:85:22 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:86:20:86:26 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:87:16:87:22 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:88:20:88:26 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:89:17:89:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:90:21:90:27 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:91:15:91:21 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:92:19:92:25 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:93:5:93:11 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:94:9:94:15 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
identityLocalStep
| test.go:114:37:114:38 | rc | Node steps to itself |
| test.go:198:27:198:29 | out | Node steps to itself |
| test.go:224:27:224:29 | out | Node steps to itself |
| test.go:249:27:249:29 | out | Node steps to itself |
| test.go:274:27:274:29 | out | Node steps to itself |
| test.go:299:27:299:29 | out | Node steps to itself |
| test.go:324:26:324:28 | out | Node steps to itself |
| test.go:349:26:349:28 | out | Node steps to itself |
| test.go:375:28:375:30 | out | Node steps to itself |
| test.go:403:28:403:30 | out | Node steps to itself |
| test.go:431:24:431:26 | out | Node steps to itself |
| test.go:463:26:463:28 | out | Node steps to itself |
| test.go:490:26:490:28 | out | Node steps to itself |
| test.go:517:27:517:29 | out | Node steps to itself |
| test.go:546:26:546:28 | out | Node steps to itself |
| test.go:571:26:571:28 | out | Node steps to itself |
| test.go:601:24:601:26 | out | Node steps to itself |
| test.go:614:15:614:21 | tarRead | Node steps to itself |
| test.go:622:3:622:7 | files | Node steps to itself |
| test.go:637:10:637:16 | tarRead | Node steps to itself |
| test.go:647:10:647:16 | tarRead | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
reverseRead
| SensitiveConditionBypassBad.go:7:5:7:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:16:5:16:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:25:5:25:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:34:5:34:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:41:5:41:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:41:35:41:35 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:49:5:49:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:56:5:56:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:63:5:63:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:70:5:70:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:77:5:77:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:84:5:84:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
reverseRead
| ConditionalBypassBad.go:9:5:9:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| ConditionalBypassGood.go:9:5:9:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:9:5:9:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:16:5:16:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:16:41:16:41 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:23:5:23:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
reverseRead
| builtin.go:115:31:115:31 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| builtin.go:124:32:124:32 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| builtin.go:133:54:133:54 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| builtin.go:142:55:142:55 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| new-tests.go:62:31:62:33 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| new-tests.go:78:18:78:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| new-tests.go:81:37:81:39 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
reverseRead
| CorsMisconfiguration.go:52:14:52:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:59:14:59:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:66:17:66:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:74:14:74:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:81:14:81:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:88:14:88:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:101:14:101:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:112:14:112:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:126:15:126:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:141:14:141:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:156:14:156:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:170:14:170:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:194:17:194:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:206:14:206:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
identityLocalStep
| CorsMisconfiguration.go:208:13:208:18 | origin | Node steps to itself |
| CorsMisconfiguration.go:235:6:235:8 | try | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
reverseRead
| WrongUsageOfUnsafe.go:34:40:34:47 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:55:40:55:47 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:77:43:77:50 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:111:47:111:54 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:211:47:211:54 | harmless | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
reverseRead
| RemoteSources.go:98:9:98:24 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
reverseRead
| pkg1/embedding.go:56:29:56:39 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| pkg1/embedding.go:57:33:57:46 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| pkg1/embedding.go:58:14:58:22 | implicit read of field embedder | Origin of readStep is missing a PostUpdateNode. |
| pkg1/embedding.go:58:31:58:42 | implicit read of field embedder | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:43:6:43:6 | t | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:46:6:46:6 | t | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:50:2:50:2 | t | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:53:6:53:7 | t2 | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:55:6:55:7 | t2 | Origin of readStep is missing a PostUpdateNode. |
| promoted.go:18:6:18:6 | t | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
reverseRead
| main.go:49:2:49:4 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| main.go:50:2:50:4 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| main.go:58:2:58:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| main.go:63:49:63:49 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:8:6:8:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:9:6:9:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:10:6:10:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:13:6:13:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
reverseRead
| Builtin.go:7:2:7:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:13:2:13:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:22:2:22:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:32:2:32:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:39:2:39:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Loading
Loading