Data source URL is invalid

[anuveyatsu]

This URL doesn't return CSV data anymore - https://github.com/datasets/population/blob/master/scripts/process.py#L10

To add "this url" is http://api.worldbank.org/countries/all/indicators/SP.POP.TOTL?format=csv

Response is:

<wb:error xmlns:wb="http://www.worldbank.org">
<wb:message id="111" key="Invalid format">The requested response format was not found.</wb:message>
</wb:error>

[rufuspollock]

@anuveyatsu it just looks like API is still there in general it is just they removed ?format=csv option and changed to downloadformat. Annoyingly though they now zip the file. Frankly i think the easiest would be just to use the json option and convert that over to whatever we need. Here's the JSON link:

http://api.worldbank.org/v2/countries/all/indicators/SP.POP.TOTL?format=json

Details

• Overview of API: https://datahelpdesk.worldbank.org/knowledgebase/articles/889392-about-the-indicators-api-documentation
• Details on API: https://datahelpdesk.worldbank.org/knowledgebase/articles/898581

Specifically https://datahelpdesk.worldbank.org/knowledgebase/articles/898581 states:

Output Format: The API supports the following four output formats.

• XML format: http://api.worldbank.org/v2/country/all/indicator/SP.POP.TOTL?format=xml

• JSON format: http://api.worldbank.org/v2/country/all/indicator/SP.POP.TOTL?format=json

• JSONP format: http://api.worldbank.org/v2/country/all/indicator/SP.POP.TOTL?format=jsonP&prefix=Getdata

• Note: For JSONP format, prefix parameter must be specified.

• JSON-stat format: http://api.worldbank.org/v2/country/all/indicator/SP.POP.TOTL?format=jsonstat

  • Note: Refer https://json-stat.org/ for more details.

So CSV is no longer there ... However, they now have downloadformat:

Download Format: The API supports the following three download formats.

• CSV Download (Downloads to ZIP file): http://api.worldbank.org/v2/country/ind/indicator/AG.AGR.TRAC.NO?source=2&downloadformat=csv
• XML Download (Downloads to ZIP file): http://api.worldbank.org/v2/country/ind/indicator/AG.AGR.TRAC.NO?source=2&downloadformat=xml
• EXCEL Download (Downloads to ZIP file): http://api.worldbank.org/v2/country/ind/indicator/AG.AGR.TRAC.NO?source=2&downloadformat=excel