Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the common etcd client in etos-iut #83

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

andmat900
Copy link
Contributor

Applicable Issues

Description of the Change

This change migrates the generic ETOS IUT Provider to the common etcd client of ETOS API.

Alternate Designs

Possible Drawbacks

Sign-off

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: Andrei Matveyeu, [email protected]

@andmat900 andmat900 requested a review from a team as a code owner October 9, 2024 09:14
@andmat900 andmat900 requested review from t-persson and fredjn and removed request for a team October 9, 2024 09:14
Comment on lines +78 to +84
if p == nil {
_, err := etcd.client.Delete(etcd.ctx, key)
if err != nil {
return 0, fmt.Errorf("Failed to delete key %s: %s", key, err.Error())
}
return 0, nil
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this added? and why in the write function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This client shall implement the io.ReadWriter interface. The ETOS API applications are designed in a way that a database client shall be easily swapped with any other client which implements this common interface.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the question is why are we adding delete in the write function. A question I also have

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If all potential database clients are supposed to conform to io.ReadWriter interface and be swappable, we have no other choice.

The io.ReadWriter interface seems to be more-suitable for a storage like a binary file, but here it requires workarounds to fit it into a key-value pair storage like etcd.

Copy link
Collaborator

@t-persson t-persson Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather we write a null value into the database rather than having this function do things that it shouldn't do.

If deletion is required then we should have a separate interface for that and in the Stop function cast the database to that interface to see if it is possible to delete.

client := h.database.Open(r.Context(), identifier)
c, canDelete := client.(database.Deleter)
if canDelete {
  err := c.Delete()
} else {
  _, err = client.Write(nil)
}

pkg/iut/v1alpha1/v1alpha1.go Outdated Show resolved Hide resolved
logger := h.logger.WithField("identifier", identifier).WithContext(r.Context())

id, err := uuid.Parse(r.URL.Query().Get("id"))
client := h.database.Open(r.Context(), identifier)

data := make([]byte, 4096)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is 4096 always goign to be enough here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally it is around 130-150 bytes.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make([]byte) should be fine here since we cannot know the size

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make([]byte) will not compile. It has to be a list of bytes that is larger than the actual data.

Copy link
Collaborator

@t-persson t-persson Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I would recommend using something io.ReadAll instead, which will allocate a small size and grow it when necessary.

I.e. data, err := io.ReadAll(client)

}
if len(dbResp.Kvs) == 0 {
err = fmt.Errorf("No key found: %s", key)
logger.Errorf("Failed to look up status request id: %s, %s", identifier, err.Error())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we really doing a lookup here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In line 190 we do a look up by reading from the database.

internal/database/etcd/etcd.go Show resolved Hide resolved
internal/database/etcd/etcd.go Show resolved Hide resolved
logger := h.logger.WithField("identifier", identifier).WithContext(r.Context())

id, err := uuid.Parse(r.URL.Query().Get("id"))
client := h.database.Open(r.Context(), identifier)

data := make([]byte, 4096)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make([]byte) should be fine here since we cannot know the size

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants