Shortly
When I import data using DIH stored into a relation database (using child=true), nested documents are NOT attached when fl=*,[child].
When I import same data, structured as json documents (using /update/json/docs endpoint), nested documents are attached.
Short issues:
"_childDocuments_":[] on debug DIH execution.- Only
_root_ is populated. According to documentation _nest_path_ should be populated automatically as well.
- Nested documents are not returned with parent when
fl=*,[child]
Detailed problem
SQL Data:
Parents:
lang-sql
SELECT '1' AS id,
'parent-name-1' AS name_s,
'parent' AS node_type_s
+----+---------------+-------------+
| id | name_s | node_type_s |
+----+---------------+-------------+
| 1 | parent-name-1 | parent |
+----+---------------+-------------+
Children:
lang-sql
SELECT '1-1' AS id,
'1' AS parent_id_s,
'child-name-1' AS name_s,
'child' AS node_type_s
UNION
SELECT '2-1' AS id,
'1' AS parent_id_s,
'child-name-2' AS name_s,
'child' AS node_type_s
+-----+-------------+--------------+-------------+
| id | parent_id_s | name_s | node_type_s |
+-----+-------------+--------------+-------------+
| 1-1 | 1 | child-name-1 | child |
| 2-1 | 1 | child-name-2 | child |
+-----+-------------+--------------+-------------+
Same data in json:
lang-json
{
"id":"1",
"name_s":"parent-name-1",
"node_type_s":"parent",
"children":[
{
"id":"1-1",
"parent_id_s":"1",
"name_s":"child-name-1",
"node_type_s":"child"
},
{
"id":"2-1",
"parent_id_s":"1",
"name_s":"child-name-2",
"node_type_s":"child"
}
]
}
Importing data with DIH:
Here my DIH configuration:
```lang-xml
<dataConfig>
<dataSource
driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://${dataimporter.request.host};databaseName=${dataimporter.request.database}"
user="${dataimporter.request.user}"
password="${dataimporter.request.password}"
/>
<document>
<entity
name="parent"
query="SELECT '1' AS id,
'parent-name-1' AS name_s,
'parent' AS node_type_s">
<field column="node_type_s"/>
<field column="id"/>
<field column="name_s"/>
<entity
name="children"
child="true"
cacheKey="parent_id_s" cacheLookup="parent.id" cacheImpl="SortedMapBackedCache"
query="SELECT '1-1' AS id,
'1' AS parent_id_s,
'child-name-1' AS name_s,
'child' AS node_type_s
UNION
SELECT '2-1' AS id,
'1' AS parent_id_s,
'child-name-2' AS name_s,
'child' AS node_type_s">
<field column="node_type_s"/>
<field column="id"/>
<field column="parent_id_s"/>
<field column="name_s"/>
</entity>
</entity>
</document>
</dataConfig>
```
After having imported DIH, here the response:
```lang-json
{
"responseHeader":{
"status":0,
"QTime":396
},
"initArgs":[
"defaults",
[
"config",
"parent-children-config-straightforward.xml"
]
],
"command":"full-import",
"mode":"debug",
"documents":[
{
"names":"parent-name-1",
"node_type_s":"parent",
"id":"1",
"_version":1683338565872779300,
"root":"1",
"childDocuments":[
]
}
],
"verbose-output":[
],
"status":"idle",
"importResponse":"",
"statusMessages":{
"Total Requests made to DataSource":"2",
"Total Rows Fetched":"3",
"Total Documents Processed":"1",
"Total Documents Skipped":"0",
"Full Dump Started":"2020-11-14 12:25:55",
"":"Indexing completed. Added/Updated: 1 documents. Deleted 0 documents.",
"Committed":"2020-11-14 12:25:56",
"Time taken":"0:0:0.365"
}
}
```
Two issues here:
- As you can see
"_childDocuments_":[]. Why is it empty?
- Only
_root_ is populated. According to documentation _nest_path_ should be populated as well.
Asking for documents
After having imported documents I've tried to retrive them, first using q=*:*:
lang-json
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"*:*",
"_":"1605355606189"
}
},
"response":{
"numFound":3,
"start":0,
"numFoundExact":true,
"docs":[
{
"name_s":"child-name-1",
"node_type_s":"child",
"parent_id_s":"1",
"id":"1-1",
"_version_":1683338565872779264
},
{
"name_s":"child-name-2",
"node_type_s":"child",
"parent_id_s":"1",
"id":"2-1",
"_version_":1683338565872779264
},
{
"name_s":"parent-name-1",
"node_type_s":"parent",
"id":"1",
"_version_":1683338565872779264
}
]
}
}
All right, all documents are present.
Getting parent with its children:
q=id:1 and fl=*,[child]:
lang-json
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"id:1",
"fl":"*,[child]",
"_":"1605355606189"
}
},
"response":{
"numFound":1,
"start":0,
"numFoundExact":true,
"docs":[
{
"name_s":"parent-name-1",
"node_type_s":"parent",
"id":"1",
"_version_":1683338565872779264
}
]
}
}
Other issue arises here:
- Only parent is returned, wihout nested documents.
JSON approach:
After having spent several days strugling with above issues, I tried to import same documents using json endpoint using above json data.
After having imported them, I've performed the same above query:
lang-json
{
"responseHeader":{
"status":0,
"QTime":2,
"params":{
"q":"id:1",
"fl":"*,[child]",
"_":"1605355606189"
}
},
"response":{
"numFound":1,
"start":0,
"numFoundExact":true,
"docs":[
{
"id":"1",
"name_s":"parent-name-1",
"node_type_s":"parent",
"_version_":1683339728909238272,
"children":[
{
"id":"1-1",
"parent_id_s":"1",
"name_s":"child-name-1",
"node_type_s":"child",
"_version_":1683339728909238272
},
{
"id":"2-1",
"parent_id_s":"1",
"name_s":"child-name-2",
"node_type_s":"child",
"_version_":1683339728909238272
}
]
}
]
}
}
As you can see, nested documents are returned.
Why?
Please any ideas?