Version 3 with cached cross chunk edges#454

Open

akhileshh wants to merge 105 commits intomainfrom pcgv3

+3,392-2,757

Commits on Dec 5, 2024

fix: cleanup ingest code
akhileshh
committed
add ttl column family
akhileshh
committed
fix: new l2 cx edge attribute
akhileshh
committed
feat: post process sv cross edges
akhileshh
committed
fix: use longer expiry for debugging
akhileshh
committed
feat(ingest): read l2 cross edges
akhileshh
committed
feat(ingest): postprocess job handling
akhileshh
committed
fix(ingest): status
akhileshh
committed
fix: timedelta import
akhileshh
committed
fix(ingest): status
akhileshh
committed
fix(ingest): use hypenated names for valid dns
akhileshh
committed
fix: rename attr; better var names
akhileshh
committed
fix: rename attr; better var names
akhileshh
committed
fix: add more docs; better var names
akhileshh
committed
fix: move cross_edges module to ingest module; only used in ingest
akhileshh
committed
fix: reduce mem use; var names; remove unused code
akhileshh
committed
fix: adds cg typehint
akhileshh
committed
fix: reduce loc
akhileshh
committed
fix: use shorter name
akhileshh
committed
feat: cache cx edges at each layer
akhileshh
committed
fix: convert array type
akhileshh
committed
fix: use atomic edges during ingest
akhileshh
committed
fix: tests
akhileshh
committed
fix: remove postprocess step
akhileshh
committed
fix: raises specific error
akhileshh
committed
fix: removes dangerous default value
akhileshh
committed
wip: read from cached edges
akhileshh
committed
wip: edits refactor
akhileshh
committed
wip: edits refactor
akhileshh
committed
fix(ingest): cache cross chunk edges from children
akhileshh
committed
feat: add unique flag
akhileshh
committed
feat: cross edges column family gcversionrule
akhileshh
committed
fix: convert input to np arrays
akhileshh
committed
fix: linting issues
akhileshh
committed
wip: edits refactor
akhileshh
committed
fix: undo gcrule changes
akhileshh
committed
fix: add mock_edges; linting issues
akhileshh
committed
feat: edits using cached cross edges
akhileshh
committed
fix: use function for dry code
akhileshh
committed
fix: mask skipped nodes
akhileshh
committed
fix: use the correct layer variable
akhileshh
committed
fix: redis pipeline for lower latency
akhileshh
committed
fix: pass redis connection
akhileshh
committed
fix: version update for deployment
akhileshh
committed
fix: status print padding
akhileshh
committed
fix: filter active edges for split, add timestamp for reading cross chunk edges
akhileshh
committed
fix: get roots no cache flag
akhileshh
committed
fix: parent and roots no cache
akhileshh
committed
fix: out edges here dont refer to edges crossing chunk
akhileshh
committed
fix: missing timestamps
akhileshh
committed
fix: consolidate neighbor nodes cx edge updates
akhileshh
committed
fix: set to list for np.array
akhileshh
committed
fix: use copy=False where possible; some cleanup
akhileshh
committed
fix: attribute type must be np.array
akhileshh
committed
fix(ingest): worker details in status
akhileshh
committed
fix: handle empty input
akhileshh
committed
fix: use empty array instead
akhileshh
committed
fix: missed time_stamp
akhileshh
committed
fix: only consolidate cx_edge writes; update per new_id
akhileshh
committed
fix: reset parent layer in loop
akhileshh
committed
fix(ingest): use get_roots with ceil=False instead of get_parents
akhileshh
committed
fix(ingest): incorrect stop_layer
akhileshh
committed
fix: add safeguard to against data corruption
akhileshh
committed
add another safeguard
akhileshh
committed
feat: log operation_id in errors
akhileshh
committed
fix: remove temp error
akhileshh
committed
add more safeguards
akhileshh
committed
fix: circular import
akhileshh
committed
fix: consider layer 2 as well
akhileshh
committed
fix(edits): incorrect order of opeartions; documentation
akhileshh
committed
feat(ingest): add tests command
akhileshh
committed
fix(edits): make sure to add reverse edges
akhileshh
committed
fix(edits): read neighbor cx edges from cache
akhileshh
committed
fix(edits): check for no cx edges; comments
akhileshh
committed
fix(edits): update neighbor cx edges in a skipped layer
akhileshh
committed
fix(edits): make sure to update all skipped neighbors
akhileshh
committed
fix(edits): ignore new ids in neighbor update
akhileshh
committed
add docs
akhileshh
committed
fix: resolve column filter ambiguity
akhileshh
committed
fix: resolve column filter ambiguity(2)
akhileshh
committed
V3 migration (#484 )
akhileshh
committed
reset version v3
akhileshh
committed
breakup long fn
akhileshh
committed
gh actions for pcgv3
akhileshh
committed
update split tests (#497 )
akhileshh
committed
segregate update nodes logic
akhileshh
committed
fix(edits): overwrite children partners when superseded by parents
akhileshh
committed
fix: unique edges always, predecing edit ts, allow same segment merge
akhileshh
committed
Bump version: 3.0.0 → 3.0.1
akhileshh
committed
fix(edits): mask all descendants when updating cx edges
akhileshh
committed
Bump version: 3.0.1 → 3.0.2
akhileshh
committed
fix(edits): use supervoxels to get the correct cross edge parents
akhileshh
committed
Bump version: 3.0.2 → 3.0.3
akhileshh
committed
fix(edits/split): filter out inactive cross edges
akhileshh
committed
fix(edits/split): filter out inactive cross edges AT EACH LAYER
akhileshh
committed
migration debug code
akhileshh
committed
use parent timestamps to lift cx edges
akhileshh
committed
make dynamic mesh dir graph specific
akhileshh
committed
fix(upgrade): use hierarchy from supervoxels
akhileshh
committed
fix(upgrade): include cx edges at node_ts explicitly
akhileshh
committed
adds job type guard, flush_redis prompts, improved status output
akhileshh
committed
fix(upgrade): include timestamps for partner supervoxel parents
akhileshh
committed
fix(upgrade): use timestamps of partners at layers > 2
akhileshh
committed
version 3.0.9
akhileshh
committed

Commits on Dec 9, 2024

feat: use mesh dir and dynamic dir from metadata
akhileshh
committed