Reputation: 13
I have setup @xyflow/react in my reactjs app to display Table lineage, where basically I have a data.json file imported with a structure of displaying parent tables, child tables, and columns inside that table, each column also has "source_column", now I was perfectly able to generate and display Table lineage, even the edges were visible
Now wanted to change it to column lineage, where Nodes are still representing the Table, but I am displaying multiple handles corresponding to each column in that table, so, suppose a table has 2 columns,
I will display those stacked in a list, and each column will have input and output handle to create edges
this will basically allow me to create a sort of many-to-many relationship where I am connecting columns within the node, now this information, is stored in each column as well in its structure "source_column", so i am trying to use that to give ids to the node handles, while giving those same ids to the edges created.
But it seems to not generating any edges then
this is where I am pushing the edges :
Object.entries(value.columns).forEach((column) => {
var columndetails = column[1];
columndetails.source_columns.forEach((sourceColumn, idx) => {
const sourceTable = columndetails.source_tables[idx];
const edgeId = `edge-${sourceTable}-${sourceColumn}-to-${key}-${column[0]}`;
const sourceId = `${sourceTable}-${sourceColumn}-output`;
const targetId = `${key}-${column[0]}-input`;
edges.push({
id: edgeId,
source: sourceId,
target: targetId,
animated: true,
style: defaultEdgeStyle,
});
});
this is where i am trying to create handles with same IDs to ensure target & source IDs are found when pushing edges
<div style={{ position: "relative", padding: "20px" }}>
{Object.entries(data.columns).map(([columnName, columnDetails], index) => {
const positionStyle = { top: `${92 + index * 33}px`, background: "#555" };
// Handles for columns that are targets (sourced from other columns)
const targetHandles = columnDetails.source_columns.map((sourceColumn, idx) => {
const sourceTable = columnDetails.source_tables[idx];
const sourceId = `${sourceTable}-${sourceColumn}-output`;
const targetId = `${data.label}-${columnName}-input`;
return (
<Handle
key={`${columnName}-input-${idx}`}
type="target"
position={handlePosition}
id={targetId}
style={positionStyle}
/>
);
});
// Handle for columns that are sources (output to other columns)
const outputHandle = (
<Handle
key={`${columnName}-output`}
type="source"
position={sourcePosition}
id={`${data.label}-${columnName}-output`}
style={positionStyle}
/>
);
return (
<React.Fragment key={columnName}>
{targetHandles}
{outputHandle}
</React.Fragment>
);
})}
</div>
This code was working earlier for Table lineage which basically uses same method but just for parent/child table linking through edges :
working code for Table Lineage :
value.child_tables.forEach((child, index) => {
edges.push({
id: `e${key}-${child}`,
source: key,
target: child,
animated: true,
style: defaultEdgeStyle,
});
if (orientation === "horizontal") {
processNode(
child,
startX + index * siblingOffset,
y + verticalSpacing,
level + 1
);
} else {
processNode(
child,
x + horizontalSpacing,
startY + index * siblingOffset,
level + 1
);
}
});
the data.json sample :
{
"MASTER_DAILY_FILE": {
"table_type": "RAW",
"child_tables": [
"BIF_INFO"
],
"parent_tables": [],
"columns": {
"account_key": {
"description": "Unique identifier for the account",
"type": "integer",
"transformation_logic": "",
"sample_value": "12345",
"source_tables": [],
"source_columns": []
},
"date_iden": {
"description": "Date identifier",
"type": "date",
"transformation_logic": "",
"sample_value": "2023-10-01",
"source_tables": [],
"source_columns": []
}
},
"table_transformation_logic": null
},
"BIF_INFO": {
"table_type": "RAW",
"child_tables": [
],
"parent_tables": [],
"columns": {
"branch_identifier": {
"description": "Unique identifier for the branch",
"type": "String",
"transformation_logic": "",
"sample_value": "12345",
"source_tables": ["MASTER_DAILY_FILE"],
"source_columns": ["account_key"]
},
"phone_no": {
"description": "Phone number of the branch",
"type": "String",
"transformation_logic": "",
"sample_value": "555-1234",
"source_tables": ["MASTER_DAILY_FILE"],
"source_columns": ["account_key"]
},
"branch_manager_name": {
"description": "Name of the branch manager",
"type": "String",
"transformation_logic": "",
"sample_value": "John Doe",
"source_tables": [],
"source_columns": []
},
"employee_name": {
"description": "Name of the employee",
"type": "String",
"transformation_logic": "",
"sample_value": "Jane Smith",
"source_tables": [],
"source_columns": []
},
"address_po": {
"description": "Indicator if the address is a PO Box",
"type": "String",
"transformation_logic": "",
"sample_value": "Y",
"source_tables": ["MASTER_DAILY_FILE"],
"source_columns": ["account_key"]
},
"po_box_num": {
"description": "PO Box number",
"type": "String",
"transformation_logic": "",
"sample_value": "123",
"source_tables": [],
"source_columns": []
},
"street_address": {
"description": "Street address of the branch",
"type": "String",
"transformation_logic": "",
"sample_value": "123 Main St",
"source_tables": [],
"source_columns": []
},
"street_address_1": {
"description": "Alternative street address of the branch",
"type": "String",
"transformation_logic": "",
"sample_value": "Main St",
"source_tables": [],
"source_columns": []
},
"branch_city": {
"description": "City of the branch",
"type": "String",
"transformation_logic": "",
"sample_value": "New York",
"source_tables": [],
"source_columns": []
},
"city_name": {
"description": "Name of the city",
"type": "String",
"transformation_logic": "",
"sample_value": "New York",
"source_tables": [],
"source_columns": []
},
"po_state_cd": {
"description": "State code for PO Box address",
"type": "String",
"transformation_logic": "",
"sample_value": "NY",
"source_tables": [],
"source_columns": []
},
"st_state_cd": {
"description": "State code for street address",
"type": "String",
"transformation_logic": "",
"sample_value": "NY",
"source_tables": [],
"source_columns": []
},
"postal_cd": {
"description": "Postal code for PO Box address",
"type": "String",
"transformation_logic": "",
"sample_value": "10001",
"source_tables": [],
"source_columns": []
},
"st_postal_cd": {
"description": "Postal code for street address",
"type": "String",
"transformation_logic": "",
"sample_value": "10001",
"source_tables": [],
"source_columns": []
},
"branch_close_dt": {
"description": "Date when the branch was closed",
"type": "Date",
"transformation_logic": "",
"sample_value": "2023-01-01",
"source_tables": ["MASTER_DAILY_FILE"],
"source_columns": ["account_key"]
},
"branch_num": {
"description": "Branch number",
"type": "Integer",
"transformation_logic": "",
"sample_value": "100001",
"source_tables": [],
"source_columns": []
}
},
"table_transformation_logic": null
}
}
i have tried a lot of things, but only nodes are visible when I try to get the column lineage, no edges are appearing, and for Table lineage it is working absolutely fine
some things i have tried to test and debug :
Upvotes: 1
Views: 283
Reputation: 990
They resolved from their end. However, here were my two mistakes.
I still had import 'reactflow/react/dist/style.css';
instead of import '@xyflow/react/dist/style.css';
Find the latest version, clean node_modules
and re install all the packages. usually this is caused from confliciting package deps.
Hope this helps!
fyi. Version which worked: 12.0.0-next.5
Came across this issue, the new @xyflow/react's interactionWidth
property seems to be causing this issue, took the courtesy of reporting the bug.
Meanwhile what worked for me to use the "old" 'reactflow' library.
So where you have,
import { ReactFlow } from "@xyflow/react";
replace with
import { ReactFlow } from "reactflow";
make sure to do a npm i reactflow
first!
Upvotes: 0