Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add storage functions, use phonenumbers #35

Closed
wants to merge 1 commit into from
Closed

add storage functions, use phonenumbers #35

wants to merge 1 commit into from

Conversation

Kariton
Copy link
Contributor

@Kariton Kariton commented Sep 8, 2023

Hey,

because the current implementation of storage.read() or storage.save() do not work well when they run simultaneously i added the storage.smembers() and storage.sadd() capabilities from redis to the RedisStorage and InMemoryStorage classes.
the later is untested, though.

smembers - list the set of stored data
sadd - add item to the end of the existing data

i have a command that fetches content from an website. this takes time.
to make sure a user never receives the same post twice i used to:

  • storage.read()
  • do stuff
  • storage_result.append(new_id)
  • storage.save()

if the user runs the same command twice in a row it might happen that both coroutines read the same data and only the last one is actually saved.

now smembers and sadd will handle the "append" and this eliminates the race condition.

i also quickly implemented phonenumbers (#29)

EDIT: typo

@Kariton
Copy link
Contributor Author

Kariton commented Sep 8, 2023

oh.
and storage.delete() is now a thing.

@Kariton
Copy link
Contributor Author

Kariton commented Sep 9, 2023

squashed

@Kariton
Copy link
Contributor Author

Kariton commented Sep 9, 2023

smembers() does now return a list instead of a set.

i think this will be more generic.

@Kariton
Copy link
Contributor Author

Kariton commented Sep 9, 2023

i donno if this is an antipattern or okay to do it this way.

WDYT?

@Kariton
Copy link
Contributor Author

Kariton commented Sep 9, 2023

i decided that the set should be returned as such. :)

@filipre
Copy link
Owner

filipre commented Sep 20, 2023

Thank you for the PR! Always nice to see contributions from others :)

I understand the need for smembers and sadd in your case but I don't like that the implementation kind of forces other storage technologies to implement those methods now as well. The motivation behind bot.storage is to provide a quick and easy way for the user to store stuff without worrying too much about file handlers, connections, etc. Anything more complicated should be hidden. But it is kind of limited I admit.

What do you think about the following approach: The storage class also provides the public attributes bot.storage.redis and bot.storage.inmemory (as well as any other technology in the future?) so anything more "complicated" can be accessed that way. Depending on the bot configuration, bot.storage.redis will be either None or the Redis instance.

storage.delete() does sound useful, I'd add it to the abstract storage definition

@Kariton
Copy link
Contributor Author

Kariton commented Sep 25, 2023

Hey,

Yeah, i was not sure if this would fit the style.
Your approach might be more feasible and would provide the necessary functionality anyway.

i'll update this PR to accommodate this.

@Kariton
Copy link
Contributor Author

Kariton commented Oct 22, 2023

this works like this:

seen_posts = c.bot.storage.redis.smembers(storage_key)

c.bot.storage.redis.sadd(storage_key, item["id"])

WDYT?

@Kariton
Copy link
Contributor Author

Kariton commented Jan 20, 2024

i have rebased the codebase and updated the deps. :)

@Kariton
Copy link
Contributor Author

Kariton commented Jun 2, 2024

see #60

@Kariton Kariton closed this Jun 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants