Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add demo showing steering of intermediate representation #204

Open
billbrod opened this issue Apr 21, 2023 · 1 comment
Open

Add demo showing steering of intermediate representation #204

billbrod opened this issue Apr 21, 2023 · 1 comment

Comments

@billbrod
Copy link
Collaborator

@ThomasYerxa , add details here on what you've done, why it didn't quite work, what else needs to be done, etc.

@ThomasYerxa
Copy link
Contributor

The goal of this experiment is to demonstrate that the steerable is equivariant w.r.t. rotations. Specifically, a rotation in the input domain should correspond to steering the coefficients by the same amount in the pyramid domain:

  • Pyramid(Rotate(Image, x)) = Steer(Pyramid(Image), x)

  • The 'steer' operation above is not exactly the 'steer' operation in the pyramid sense, which simulates pyramid responses to a single image at different orientations. This steer simulates the pyramids response to a shifted image.

  • A simple demonstration attempt is as follows:

      1.  Initialize a pyramid with 2 orientations (horizontal and vertical)
      2. Take an input image and rotate it by 90 degrees
      3. Push the original and rotated versions through the pyramid
      4. Show the coefficients are the same but have swapped orientation bands
    
  • I have tried doing a verison of this in the 'steerpyr_equivariance' branch in the examples/03_Steerable_Pyramid.ipynb notebook. Rather than rotate by 90 degrees I tried rotating by 45 degrees (and using a pyramid with more orientation bands including an oblique at 45 degrees).

  • another detail: I apply a circular mask to the image before rotating (this seemed the right thing to do, allows for rotating at intermediate angles without introducing quite as many artifacts)

  • Its possible I've made an implementation mistake in checking for the correspondence between bands, but I didn't achieve a satisfactory result. the code in this notebook is just a starting point for this type of demonstration.

  • another possibility is that the deviations are just due to the interpolation that needs to happen when rotating an image by an angle that isn't a multiple of 90 degrees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants