- Issue created by @guignonv
- 🇫🇷France guignonv
New proposition:
Only support 2 aggregation modes:
1. each data source adds properties to the entity of the previous source(s),
either because the entity data from both sources share the same identifier or
the new data source entity identifier correspond to a specified field value of
(one of) the previous source(s) entity (ie. "chaining").2. each data source brings its own set of entities. Therefore, entity
identifiers should all differ from one data source to another. If it is not
the case, data source entity identifiers could be prefixed by xnttmulti
using a "virtual group prefix" on a data source-basis.For the first case, this module assumes the first data source holds all the
entity identifiers and can be used for pagging management. If an identifier is
not present in the first data source but in one (or more) of the next data
sources, the corresponding entity will be discarded. For filters, each
data source will be provdied the filters and should ignore irrelevant filters.
After xnttmulti has aggregated entity data, it will perform an additional
filtering to apply user filtering on aggregated data as well.For the second case, only data sources sharing the same group prefix will be
aggregated together like in the first case. The first data source of each group
will be considered as the group "reference" and processed just like in the first
case for aggregation and pagging management. Entities from data sources without
group prefix will only be considered and aggregated if their identifiers match
an earlier group prefix and an existing entity in the corresponding group
reference. There is a special case when the first(s) data source(s) is (are) not
in a group. Then, xnttmulti will use the first case behavior and only use the
next specified group as filters on each data source that has a group.By default, xnttmulti will only consider entities from a group data source if
they actually have the prefix in their identifier. Since some data source may
not use a prefix in their entity identifier, it is also possible to add as a
"virtual" prefix with xnttmulti. In that case, any identifier from a given data
source will be considered and prefixed by each group prefix. It means that if
more than one virtual prefix is specified for the first time for a given source,
its data will be virtually duplicated for each prefix (ie. 3 virtual prefixes,
3 times the same data but with 3 different prefixed identifiers).When an entity is aggregated from several data sources (same identifier or with
chaining), it is possible to choose the way to merge the last data source:
either without override and only new fields are added, or with override and any
field from the last source will replace previous existing values, except for
entity identifier field or chaingin field.Let's consider the following examples:
* Example 1:
- Data source "alpha":
- Entity 1: id='A001', field1='abc'
- Entity 2: id='A002', field1='def', field4='B002'
- Entity 3: id='X003', field1='ghi'- Data source "beta":
- Entity 1: id='B001', field1='jkl', field2='def'
- Entity 2: id='B002', field1='mno', field2='ghi', field3='B001', field4='X003'- Data source "gamma":
- Entity 1: id='A001', field1='pqr', field2='def'
- Entity 2: id='B002', field1='stu', field2='fed'
- Entity 3: id='X003', field1='vwx', field2='xyz'Examples of settings and the corresponding sets of aggregated (final) data:
- Aggregate all, no group, no override:
- Entity 1: id='A001', field1='abc', field2='def'
- Entity 2: id='A002', field1='def', field4='B002'
- Entity 3: id='X003', field1='ghi', field2='xyz'- Aggregate all, no group, with override:
- Entity 1: id='A001', field1='pqr', field2='def'
- Entity 2: id='A002', field1='def', field4='B002'
- Entity 3: id='X003', field1='vwx', field2='xyz'- Aggregate all, no group, with override, with gamma identifier as 'field4':
- Entity 1: id='A001', field1='abc'
- Entity 2: id='A002', field1='stu', field2='fed', field4='B002'
- Entity 3: id='X003', field1='ghi'- Aggregate by group prefix alpha='A', beta='B', gamma=none, no override:
- Entity 1: id='A001', field1='abc', field2='def'
- Entity 2: id='A002', field1='def', field4='B002'
- Entity 3: id='B001', field1='jkl', field2='def'
- Entity 4: id='B002', field1='mno', field2='ghi', field3='B001', field4='X003'- Aggregate by group prefix alpha='A,X', beta='B', gamma=none, no override:
- Entity 1: id='A001', field1='abc', field2='def'
- Entity 2: id='A002', field1='def', field4='B002'
- Entity 3: id='X003', field1='ghi', field2='xyz'
- Entity 4: id='B001', field1='jkl', field2='def'
- Entity 5: id='B002', field1='mno', field2='ghi', field3='B001', field4='X003'- Aggregate by group prefix alpha='A', beta='B', gamma='B,X', with override:
- Entity 1: id='A001', field1='abc'
- Entity 2: id='A002', field1='def', field4='B002'
- Entity 3: id='X003', field1='vwx', field2='xyz'
- Entity 4: id='B001', field1='jkl', field2='def'
- Entity 5: id='B002', field1='stu', field2='fed', field3='B001', field4='X003'* Example 2:
- Data source "alpha":
- Entity 1: id='001', field1='abc'
- Entity 2: id='002', field1='def', field4='002'
- Entity 3: id='003', field1='ghi'- Data source "beta":
- Entity 1: id='001', field1='jkl', field2='def'
- Entity 2: id='002', field1='mno', field2='ghi', field3='001', field4='003'- Data source "gamma":
- Entity 1: id='001', field1='pqr', field2='def'
- Entity 2: id='002', field1='stu', field2='fed'
- Entity 3: id='003', field1='vwx', field2='xyz'Examples of settings and the corresponding sets of aggregated (final) data:
- Aggregate by group prefix alpha='A', beta='B', gamma=none, no override, with virtual prefix:
- Entity 1: id='A001', field1='abc'
- Entity 2: id='A002', field1='def', field4='002'
- Entity 3: id='A003', field1='ghi'
- Entity 4: id='B001', field1='jkl', field2='def'
- Entity 5: id='B002', field1='mno', field2='ghi', field3='001', field4='003'- Aggregate by group prefix alpha='A', beta='B', gamma='A,B', with override, with virtual prefix:
- Entity 1: id='A001', field1='pqr', field2='def'
- Entity 2: id='A002', field1='stu', field2='fed', field4='002'
- Entity 3: id='A003', field1='vwx', field2='xyz'
- Entity 4: id='B001', field1='pqr', field2='def'
- Entity 5: id='B002', field1='stu', field2='fed', field3='001', field4='003'- Aggregate by group prefix alpha=none, beta='002', gamma=none, with override, with gamma identifier as 'field4' (no virtual identifier):
- Entity 1: id='001', field1='abc'
- Entity 2: id='002', field1='vwx', field2='xyz', field3='001', field4='003'
- Entity 3: id='003', field1='ghi' - Status changed to Fixed
over 1 year ago 11:54am 2 August 2023 Automatically closed - issue fixed for 2 weeks with no activity.