Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new texture model #200

Open
billbrod opened this issue Apr 3, 2023 · 1 comment
Open

Add new texture model #200

billbrod opened this issue Apr 3, 2023 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@billbrod
Copy link
Collaborator

billbrod commented Apr 3, 2023

This is a good issue for a summer intern or new grad student.

The PortillaSimoncelli texture model implementation is a bit finicky, because we're trying to match the matlab implementation, which is a bit finicky. Our general recommendation is that people should use a new implementation that captures the same statistics, but does so in a modern way. It should compute the cross- and auto-correlations across scale and orientation. One thing that might be tricky is the phase-doubling trick the authors use to make correlations work across scales.

@billbrod billbrod added enhancement New feature or request good first issue Good for newcomers labels Apr 3, 2023
@nikparth
Copy link
Collaborator

I'd say a big issue with the current implementation is the fact that in computing the various statistics there are a lot of places where for loops are done over all scales/orientations to compute statistics such as autocorrelation per band. To make this really scalable with batches, you would also need to figure out how to vectorize those operations. That would be primary focus since that would have the biggest impact on speed. In addition, adding the ability to process a batch will require careful stepping through the code and either modifying the operations to handle the batch dimesion or maybe utilizing the in-built torch vmap: https://pytorch.org/tutorials/prototype/vmap_recipe.html. Using vmap may be the cleanest way because we can still maintain the intuition of computing these statistics per image but then vmap that operation across the batch dimension.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants