Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SYNPY-1337] Adding note about unique .db file #1016

Merged
merged 3 commits into from
Nov 27, 2023
Merged

[SYNPY-1337] Adding note about unique .db file #1016

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
55e1df3
Adding note about unique .db file
BryanFauble Nov 27, 2023
56d66b8
Update script to include sending an email on migration finish
BryanFauble Nov 27, 2023
af1f516
Run pre-commit
BryanFauble Nov 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions docs/articles/data_storage.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,10 @@ Once the storage location is known, the first step to migrate the project or fol
of its contents using the
`index_files_for_migration <synapseutils.html#synapseutils.migrate_functions.index_files_for_migration>`__ function, e.g.

When specifying the `.db` file for the migratable indexes you need to specify a `.db` file that does not already exist
for another synapse project or folder on disk. It is the best practice to specify a unique name for the file by including
the synapse id in the name of the file, or other unique identifier.

.. code-block::

import synapseutils
Expand All @@ -149,7 +153,7 @@ of its contents using the
# nothing needs to exist at this path, but it must be a valid path on a volume with sufficient
# disk space to store a meta data listing of all the contents in the indexed entity.
# a rough rule of thumb is 100kB per 1000 entities indexed.
db_path = '/tmp/foo/bar.db'
db_path = '/tmp/foo/syn123_bar.db'

result = synapseutils.index_files_for_migration(
syn,
Expand Down Expand Up @@ -207,7 +211,7 @@ Putting all the migration pieces together
# # nothing needs to exist at this path, but it must be a valid path on a volume with sufficient
# # disk space to store a meta data listing of all the contents in the indexed entity.
# # a rough rule of thumb is 100kB per 1000 entities indexed.
db_path = os.path.expanduser("~/synapseMigration/my.db")
db_path = os.path.expanduser(f"~/synapseMigration/{my_synapse_project_or_folder_to_migrate}_my.db")

syn = synapseclient.Synapse()

Expand Down Expand Up @@ -253,7 +257,7 @@ Putting all the migration pieces together
The result of running this should look like
.. code-block::

Indexing: syn123 for migration to storage_id: 11111 at: /home/user/synapseMigration/my.db
Indexing: syn123 for migration to storage_id: 11111 at: /home/user/synapseMigration/syn123_my.db
Indexing result: {'INDEXED': 100, 'MIGRATED': 0, 'ALREADY_MIGRATED': 0, 'ERRORED': 0}
Migrating files...
Migration result: {'INDEXED': 0, 'MIGRATED': 100, 'ALREADY_MIGRATED': 0, 'ERRORED': 0}
Expand Down
Loading