bulk_search API returns incomplete / partial data using Chunked Transfer Encoding

[fviernau]

Scenario

Neither with curl, nor with reqests lib (Python) nor with OkHttp (Kotlin) it is possible
to get the complete data of the response, for package: pkg:maven/com.google.guava/guava@19.0.
It seems that the server attempts to use Chunked transfer encoding.

It reproduces for both, API v1 + v2.

Note: There are other packages, for which the problem does not happen.

Reproduce with curl

curl 'https://public.vulnerablecode.io/api/packages/bulk_search' \
  -H 'Content-Type: application/json; charset=utf-8' \
  --data '{"purls":["pkg:maven/com.google.guava/guava@19.0"]}'

gives

curl: (18) transfer closed with 64255 bytes remaining to read

Reproduce with Python

import requests

url = 'https://public.vulnerablecode.io/api/packages/bulk_search'
myobj = { 
  "purls": [
    "pkg:maven/com.google.guava/guava@19.0"
    ]
}

x = requests.post(url, json = myobj, stream=False)

print(x.text)

gives:

requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(97975 bytes read, 64255 more expected)', IncompleteRead(97975 bytes read, 64255 more expected))

[Misoding]

I will take a look on this issue, my hypotesis is that this problem is based on out of memory of container due to high number of vulnerabilities of this old package. I will debug this problem locally to identify the root cause and to return with a solution :)

[ionfwsrijan]

Hi there,

I’m Srijan Jaiswal, and I’m eager to contribute to this project as part of my open-source contribution. I have reviewed this issue and feel confident that I can provide a solution since I have experience working with similar technologies in my past projects. Could you please assign this issue to me? I’d love to work on it and contribute to the project.

Best regards,
Srijan Jaiswal