Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port CenterPillar to Keras Core #2003

Merged
merged 16 commits into from
Aug 29, 2023

Conversation

ianstenbit
Copy link
Contributor

@ianstenbit ianstenbit commented Aug 1, 2023

TODOs (all done now -- sending for review)

  • Avoid _core versions of voxel_utils
  • Figure out where to put segment_max
  • Use keras_core.backend.name_scope
  • Reorder components in center_pillar.py

@ianstenbit ianstenbit marked this pull request as ready for review August 14, 2023 20:42
@ianstenbit
Copy link
Contributor Author

/gcbrun

@ianstenbit
Copy link
Contributor Author

/gcbrun

Copy link
Contributor

@jbischof jbischof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ianstenbit I thought this model wasn't possible to port due to custom tf ops and I obviously see a lot of TF code remaining. Is the idea that this is a TensorFlow-only model that works with Keras Core? If so, is this something we can flag for users?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file still has a LOT of tensorflow code, is that intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep! This lives outside of the model and runs in tf.data

def point_to_voxel_coord(
point_xyz: tf.Tensor, voxel_size: Sequence[float], dtype=tf.int32
) -> tf.Tensor:
def point_to_voxel_coord(point_xyz, voxel_size, dtype=tf.int32):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function also has a lot of TF

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Same here -- this is for tf.data)

@ianstenbit
Copy link
Contributor Author

@ianstenbit I thought this model wasn't possible to port due to custom tf ops and I obviously see a lot of TF code remaining. Is the idea that this is a TensorFlow-only model that works with Keras Core? If so, is this something we can flag for users?

This does work with other backends, but requires data encoding in a tf.data pipeline, so that code is all still written in TF.

Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Feel free to merge whenever, just left some random low level comments for things that popped out.

keras_cv/backend/__init__.py Outdated Show resolved Hide resolved
keras_cv/layers/object_detection_3d/heatmap_decoder.py Outdated Show resolved Hide resolved
keras_cv/layers/object_detection_3d/voxel_utils.py Outdated Show resolved Hide resolved
keras_cv/layers/object_detection_3d/voxel_utils.py Outdated Show resolved Hide resolved
keras_cv/layers/object_detection_3d/voxelization_test.py Outdated Show resolved Hide resolved
@ianstenbit
Copy link
Contributor Author

/gcbrun

@ianstenbit
Copy link
Contributor Author

/gcbrun

@ianstenbit
Copy link
Contributor Author

/gcbrun

@ianstenbit ianstenbit merged commit b9d5cda into keras-team:master Aug 29, 2023
6 of 9 checks passed
@ianstenbit ianstenbit deleted the core-centerpillar branch August 29, 2023 17:57
ghost pushed a commit to y-vectorfield/keras-cv that referenced this pull request Nov 16, 2023
* Port CenterPillar to Keras Core

* Switch back to backbone API

* Restructure center_pillar.py

* Undo some mistaken changes

* More fixes -- tests currently failing

* Test passing across backends

* Remove local segment_max op

* Finish port + tests passing

* Fix tests + format

* Remove numpy behavior

* Moar test fixes

* Use smaller backbone

* Review comments

* Add 3D OD models to multi-backend CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants