Hello all,

Speaking of dbnomics, I have noticed in my version of gretl (2018a for Windows) that i have dbnomics following the path help/look for addons/dbnomics 0.1

However, I tried to install it but it did not work.  I got a responce indicating dbnomic could’n be found.

Any idea what is going wrong?

Thanks for Information in advance!

Regards,

Waldemar

Von: Riccardo (Jack) Lucchetti
Gesendet: Mittwoch, 4. Juli 2018 10:02
An: Gretl users mailing list
Betreff: [Gretl-users] Datahub

Dear all,

Allin and I are working on automated data download from dbnomics (if you

don't know what I'm talking about: https://db.nomics.world). This led both

of us to using JSON object more systematically than we used to, and I have

to say JSON is very nice.

During my experiments, I wrote a little function for automatically

downloading datasets from DataHub (https://datahub.io). A small example

script follows:

<hansl>

set verbose off

function string datahub_get(string URL, string filename[null])

     s = readfile(URL)

     printf "reading %s...\n", URL

     flush

     printf "Dataset name: %s\n", jsonget(s, "$.title")

     resources = jsonget(s, "$.resources..datahub.type")

     got = 0

     loop i = 1 .. nelem(resources) --quiet

         string r = strstrip(jsonget(s, sprintf("$.resources[%d].datahub.type", i-1)))

         if r == "derived/csv"

             hit = i-1

             got = 1

             break

         endif

     endloop

     if got

         if exists(filename)

             ret = filename

         else

             fname = sprintf("%04d.csv", randgen1(i,0,9999))

             if $windows

                 ret = sprintf("@dotdir\\%s", fname)

             else

                 ret = sprintf("@dotdir/%s", fname)

             endif

         endif

         printf "data URL found, reading...\n"

         flush

         path = jsonget(s, sprintf("$.resources[%d].path", hit))

         c = readfile(path)

         outfile "@ret" --quiet --write

         print c

         outfile --close

     else

         printf "No data found\n"

         ret = ""

     endif

     return ret

end function

URL = "https://pkgstore.datahub.io/core/pharmaceutical-drug-spending/19/datapackage.json"

fname = datahub_get(URL)

print fname

open "@fname"

</hansl>

Of course the function above can be enhanced in many ways, but the gist

is: once you have the URL to the dataset, the function will download the

data to a CSV file (you can supply a name if you want, otherwise a random

one will be generated for you). At that point, you may just open the CSV

file.

How do you get the URLs for the datasets you want? Easy:

* Navigate to the page of the dataset you want (for example,

https://datahub.io/core/co2-ppm)

* scroll to the bottom and copy the URL corresponding to the

"Datapackage.json" button. In most cases, it's just the page URL with

"/datapackage.json" appended at the end (for example,

"https://datahub.io/core/co2-ppm/datapackage.json")

* run the function datahub_get() with the URL string as first parameter;

you maty supply an optional second parameter if you want to save the CSV

file to a location of your choice.

Enjoy!

-------------------------------------------------------

   Riccardo (Jack) Lucchetti

   Dipartimento di Scienze Economiche e Sociali (DiSES)

   Università Politecnica delle Marche

   (formerly known as Università di Ancona)

   r.lucchetti@univpm.it

   http://www2.econ.univpm.it/servizi/hpp/lucchetti

-------------------------------------------------------