Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latent space documentation #15

Open
wants to merge 28 commits into
base: main
Choose a base branch
from
Open

Conversation

punkduckable
Copy link
Collaborator

I added documentation (comments + doc strings + type hints + formatting) to latent_space.py

The code is functionally almost identical, save for a few minor tweaks:

Removed the MultiHeadAttention activation function option. This isn't really an activation function, even though it is listed as such in the torch api. It is used to implement an attention layer in a transformer block. It introduces a whole new set of parameters (for the key, query, and value weight matrices) and is really designed to map finite sequences to finite sequences. I do not think it makes sense to include this in the MLP class as an activation option, so I removed it.

Removed apply_attention. Since there is no more multiheadedattention activation option, this function should be removed.

Otherwise, all changes are cosmetic (e.g., comments + formatting).

I also fixed a small bug in gplasdi.py: np.Inf no longer exists. It is deprecated. It has been replaced with np.inf. I fixed this.

dreamer2368 and others added 28 commits September 16, 2024 13:45
I also removed the multheadedattention activation type (plus the associated apply_attention function) because they did not make sense in this class.

I have now documented the MLP class, but need to add documentation to the autoencoder class.
I added comments + doc strings to the Autoencoder class.
gplasdi had an instance of np.Inf (which is depricated). I also fixed a typo in latent_spaces.py
@dreamer2368
Copy link
Collaborator

dreamer2368 commented Oct 24, 2024

This PR will be rebased and merged after PR #11.
There are code conflicts in PR, which happens after other PRs get merged to main branch. Since this is already based on sphinx branch (PR #11), technically it does not have a conflict, only that github does not automatically resolve the past commits. Once PR #11 is merged, rebasing the last 3 commits from the (merged) main branch would not require any editing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants