meaningfool

Activity Log - 2025-11-28

Analysis

  • Implemented Robust Graph Layout Algorithm for Data Lineage Visualization:

    • Developed a custom “Single Pass Sink-Anchored” algorithm to replace Dagre for handling complex lineage graphs with invalid flows and cycles
    • Key design decision: Traverse upstream (Target → Source) from “Sinks” (Outputs and Dead-end Steps) to naturally handle Right-to-Left visual flow
    • Introduced “Selective Reversal” to virtually reverse invalid edges (Output->Step, Step->Input) so they participate in flow calculation
    • Handled degenerate cases: Input-Only graphs (early exit to Layer 0), strict Input separation (force Inputs to MaxLayer+1 when needed)
    • Solved the Floating Cycles problem (isolated A->B->C->A loops with no sinks) using a “Greedy Deterministic Leader” approach: pick lowest-ID unvisited node as Pseudo-Sink, anchor to Layer 1, unroll upstream
    • Switched to Bezier Curves (curvature 0.35) to reduce visual clutter in dense graphs with overlapping edges
    • Implemented proper RTL flow by swapping handle positions and inverting backward edge logic

  • Established Two-Layer Testing Strategy:

    • Layer 1 (Algorithm Logic): Add data-layer attribute to nodes for E2E tests to verify computed layer indices directly
    • Layer 2 (Visual Rendering): Assert relative X-coordinates to verify correct rendering
    • Decouples logic verification from pixel-perfect rendering, making tests more robust

  • AI Process Improvement - New CLAUDE.md Established:

    • Created CLAUDE.md for the lineage-graph-app with Spec-Driven Development workflow
    • Defined communication protocol: Question → Answer only; Action request → Explain first, then do; Specific request → Do exactly that
    • Established E2E-first testing policy with TDD and data-testid selectors

Raw Commits

Repository: meaningfool/lineage-graph-app Commit: [0762f338d0bf0e21ee566ebac5ce3ede43bcf056] Merge feat/graph-layout: Implement Robust Graph Layout Algorithm Description: This feature introduces a custom ‘Single Pass Sink-Anchored’ algorithm to handle complex data lineage graphs, including invalid flows and cycles.

Key Learnings & Implementation Details:

  1. Single Pass Sink-Anchored Algorithm:

    • Anchors traversal at ‘Sinks’ (Outputs and Dead-end Steps).
    • Traverses Upstream (Target -> Source) to naturally handle Right-to-Left visual flow.
    • Handles degenerate cases like Input-Only graphs (Early Exit) and strict input separation.

  2. Floating Cycles (Pseudo-Sink Strategy):

    • Solves the problem of isolated cycles (e.g., A->B->C->A) having no sinks.
    • Uses a ‘Greedy Deterministic Leader’ approach: picks the lowest-ID unvisited node as a ‘Pseudo-Sink’, anchors it to Layer 1, and unrolls the cycle upstream.

  3. Visual Improvements:

    • Switched to Bezier Curves (curvature 0.35) to reduce visual clutter in dense graphs.
    • Implemented Right-to-Left flow by swapping handle positions and inverting backward edge logic.

  4. Testing Strategy:

    • Adopted a two-layer testing strategy: verifying logic via ‘data-layer’ attributes and visual rendering via relative coordinates.
    • Addressed E2E flakiness by tuning timeouts for heavy graph operations.

Repository: meaningfool/lineage-graph-app Commit: [e015da97eafa736a7d8723aa8d45b3ed5c31029d] docs: condense learnings.md into key algorithm insights