Weird "Could not open the datasource at …" error

[torotil]

When I try to import he data using this guide I always get an "Could not open the datasource at …" error for the first file of the boundary-line data.

Strangely the problem goes away if I add 2 lines directly to manage.py:

from django.contrib.gis.gdal import DataSource

# …
     DataSource('…/Data/GB/county_electoral_division_region.shp')
# …

It seems that the first call to DataSource has some (in that case helpful) side-effects.

[dracos]

"Could not open the datasource" implies you are not giving the correct path to the data on the command line. It looks like the command in the docs only works if you have moved all the .shp files into the same directory, it's not what the zip file contains by default. I've updated the docs, even if that's not the cause of your issue, but I guess that's probably it.

[torotil]

Yes that would be the easy explanation. But the path is indeed correct. And it does work perfectly fine if I add the above code to manage.py -- Note that with "working perfectly fine" I don't mean just the added lines but the whole import!

So to make it more clear:

1. Try the import command -> fail with the above error.
2. Add lines to manage.py
3. Try the very same command -> all shape-files are imported.

[dracos]

So I can't think what to suggest, sorry. DataSource is just a wrapper for the underlying GDAL library, so I don't really know what it could be doing. What OS, what version of Django, what version of GDAL are you using? It seems to be okay on e.g. Debian wheezy, GDAL 1.9, Django 1.8.

[torotil]

This is

Django==1.9.4
GDAL==1.10.1

on debian jessie.

I don't think it's that urgent issue if noone else sees this. And even if: There is a workaround.

I've found the workaround while debugging: First I checked if the paths were correct. I even tried with absolute paths. My next attempt was to move the DataSource() call up the stack. That's when I found that it suddenly works if I put those lines into manage.py.

[torotil]

So far I've tracked it down to django's check_resolver(). When I call DataSource() before those checks everything works - otherwise it fails.

[dracos]

Just to note that on jessie here, with (system) GDAL 1.10.1 (no python package installed) and Django 1.8.12 or Django 1.9.5, it works okay for me. I wonder what's different!

[torotil]

My first guess was that the working directory is changed at some point. But that would only be relevant for relative URLs. I might take some time to take a closer look later today.

[WorldChgMe]

I had the exact same problem as torotil and the same solution worked for me. I'm using Amazon Web Services with the following
gdal version = 1.9.0
django version = 1.9.6
linux version = Debian 7.8

Thank you for the workaround torotil.

[chris48s]

I can at least confirm that this is not a problem which is specific to mapit. We've observed this same behaviour from DataSource() in a completely different project and had to hack around the issue by doing something like:

try:
    ds = DataSource(kml)
except GDALException:
    ds = DataSource(kml)

# do stuff with ds

[dracos]

https://community.bitnami.com/t/rebuild-gdal-with-expat-xml-parser/41762/2 seems to be the same too.

I've replicated the issue in jessie.

[dracos]

Okay, this is due to I think a bug in Django prompted by our import of gdal from osgeo in areas.py. I've reported it at https://code.djangoproject.com/ticket/27103 and put in a hopeful fix at django/django#7137 though as that'll presumably only go in the next version, we should work around it here somehow, PR at #260

[torotil]

Ah yes. The merits of import-time side-effects. (Most often disguised using decorators.)

Thanks for tracking this down!

[dracos]

#260 has been merged.