Japanese Virtuoso SPARQL EndPoint times out very quickly

Question

I am trying to use the following query on Japanese dbpedia SPARQL Endpoint

select ?s  (group_concat(?album_ja ; separator = "|") AS ?name_album_ja_csv)  where{
   values ?sType { dbpedia-owl:Song
                       dbpedia-owl:Single
    } .
      ?s a ?sType .
      ?s (dbpedia-owl:album|^dbpedia-owl:album)* ?albums;rdfs:label ?album_ja      
    }group by ?s order by ?s offset 0 limit 10

but I get this error Virtuoso 42000 Error The estimated execution time 1005 (sec) exceeds the limit of 400 (sec). Almost any query involving group by has this problem. Is this a server problem? Is my query inefficient? How can I get around it?

Two things seem likely. 1) You're not *using* ?o or ?oType anywhere, so they simply multiply the number of results you've got greatly. (see [sparql query running forever](http://stackoverflow.com/q/25304721/1281433)) 2) The property path `(dbpedia-owl:album|^dbpedia-owl:album)*` is probably estimated to be pretty complex. Do you really need to follow arbitrary length paths in both directions? Would `?` work rather than `*`? Or `{0,1}` (which isn't part of standard SPARQL, but I think Virtuoso accepts it)? — Joshua Taylor, Aug 22 '14 at 16:48

score 1 · Accepted Answer · answered Aug 22 '14 at 16:52

1

I'm not sure exactly what you're trying to do, but since ?o isn't used in the results, you can get rid of it. Even after you do that, though, you'll still have the same problem. You need to change the property path somehow. I don't think that you actually need arbitrary length paths in both directions. You could use ? instead of * to say "path of length 0 or 1" and thus:

select ?s
       (group_concat(?album_ja ; separator = "|") AS ?name_album_ja_csv)
where {
  values ?sType { dbpedia-owl:Song dbpedia-owl:Single }
  ?s a ?sType .
  ?s (dbpedia-owl:album|^dbpedia-owl:album)? ?album_ja      
}
group by ?s
limit 100

SPARQL results

Note that a path of length 0 is going to be a link to itself, so one value of ?album_ja is always going to be the same as ?s. Is this really what you want?

answered Aug 22 '14 at 16:52

Joshua Taylor

80,876
9
135
306

As you mentioned, The problem was (dbpedia-owl:album|^dbpedia-owl:album)* when I replaced it with (dbpedia-owl:album|^dbpedia-owl:album){1} – user1848018 Aug 22 '14 at 17:36
There's no reason to use `{1}`; that just says "exactly one time", which is the same as `dbpedia-owl:album|^dbpedia-owl:album`. – Joshua Taylor Aug 22 '14 at 17:38
The problem is now the offset. for offset 9990 limit 10, it return data but when I do offset 10000, it returns nothing. The Japanese dbpedia has the limit of 10000, I know but isn't hat offset is supposed to do here? – user1848018 Aug 22 '14 at 17:38
offset and limit only make sense if you're ordering the results. See [my answer](http://stackoverflow.com/a/25147648/1281433) to [How to resolve the execution limits in Linkedmdb](http://stackoverflow.com/q/25141247/1281433). – Joshua Taylor Aug 22 '14 at 17:39
That is what I thought but without {1} at the end of (dbpedia-owl:album|^dbpedia-owl:album), I will get this error SPARQL compiler: Internal error: sparp_tree_full_clone_int(): unsupported type of expression – user1848018 Aug 22 '14 at 17:40
Regarding the offset comment, yes I did follow your instruction and did use "order by". Look at my original query. It has group by ?s order by ?s offset 0 limit 10, I just changed the offset to be 10000 – user1848018 Aug 22 '14 at 17:42
Hm, I see what you mean about Virtuoso accepting `xxx{1}`, but not `xxx`. Just a strange bit of Virtuoso, I guess. I see that you've updated your question, but [your **original query**](http://stackoverflow.com/revisions/25451330/1) did not have `order by`. – Joshua Taylor Aug 22 '14 at 17:46
Yes I did, so what should I do for the offset thing? the offset 9990 limit 10 works fine but offset 10000 limit 10 return nothing. is it a bug? – user1848018 Aug 22 '14 at 17:48
If you use something like `limit 10 offset 10000` you get the message `Virtuoso 22023 Error SR353: Sorted TOP clause specifies more then 10010 rows to sort. Only 10000 are allowed. Either decrease the offset and/or row count or use a scrollable cursor`. I think you'll need to do one of those things. At this point, though, that's really a different question. See [this answer](http://stackoverflow.com/a/20939857/1281433) and its comments. – Joshua Taylor Aug 22 '14 at 17:48
Sometimes I get the error you showed but this time I just got no values. Strange. Thanks a lot for always being helpful – user1848018 Aug 22 '14 at 17:52

Japanese Virtuoso SPARQL EndPoint times out very quickly

1 Answers1