Incoming requests: bad parsed query value

[benjamin-thomas]

{
    "type": "application",
    "platform": "node",
    "source-directories": [
        "src"
    ],
    "gren-version": "0.5.3",
    "dependencies": {
        "direct": {
            "gren-lang/core": "6.0.1",
            "gren-lang/node": "5.0.3",
            "gren-lang/url": "5.0.0"
        },
        "indirect": {}
    }
}

---

When a request comes in, the parsed URL should look like this:

-- Url.fromString "http://localhost:4000/blog/123?q=0&r=1#tag"
Just { fragment = Just "tag", host = "localhost", path = "/blog/123", port_ = Just 4000, protocol = Http, query = Just "q=0&r=1" }

However, the query attribute comes with an extra ?, making the related query parser fail.

{ fragment = Nothing, host = "localhost", path = "/blog/123", port_ = Just 4000, protocol = Http, query = Just "?q=0&r=1" }

The problem seems to be here:

node/src/Gren/Kernel/HttpServer.js

Line 47 in db725ad

__$urlQuery: url.search,

In node, an extra char appear in the parsed output of query and hash:

> new URL("http://localhost:8080/blog/1?a=123&b=456#tag")
URL {
  href: 'http://localhost:8080/blog/1?a=123&b=456#tag',
  origin: 'http://localhost:8080',
  protocol: 'http:',
  username: '',
  password: '',
  host: 'localhost:8080',
  hostname: 'localhost',
  port: '8080',
  pathname: '/blog/1',
  search: '?a=123&b=456',
  searchParams: URLSearchParams { 'a' => '123', 'b' => '456' },
  hash: '#tag'
}

I wanted to try a change locally, by pointing my gren-lang/node to a localfile, but couldn't compile. Is it possible to play with kernel code locally to experiment?

I think we should run url.search.slice(1) and url.hash.slice(1) to conform to the historical parse result.

[blaix]

It looks like there's two problems: the extra ? as you pointed out, which is fixed by passing the full url string to gren to parse, but also the browser doesn't send the hash fragment with the http request. I'm going to spend a little more time confirming this, but I believe we may have to accept HttpServer.Request.url will never have a fragment.

[benjamin-thomas]

Oh, of course! I completely forgot that the hash is not sent by the client!

I double checked this, just in case

Also, it's quite poorly worded in RFC3986, but I found RFC7230 makes this clearer in section 5.1

The target URI excludes the reference's fragment component, if any,
since fragment identifiers are reserved for client-side processing

It's a bit of a HTTP gotcha, I guess there's no good solution here and a brief mention in the docs would be good enough.

[blaix]

Thanks for the details! I'll fix the extra ? and update the docs soon.

[blaix]

Test for this bug is here: gren-lang/integration-tests#1

Fix is here: #30