Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sptensor constructor breaks with int overflow for large tensor #305

Open
jeremy-myers opened this issue Jun 28, 2024 · 0 comments
Open

Sptensor constructor breaks with int overflow for large tensor #305

jeremy-myers opened this issue Jun 28, 2024 · 0 comments
Assignees

Comments

@jeremy-myers
Copy link
Collaborator

jeremy-myers commented Jun 28, 2024

Issue

The constructor for sptensor fails an assertion when the tensor size is large. It's important to note that the shape isn't allocated, simply checked for consistency.

Example

For size = (50000000, 50000000, 100, 50000) and nnzs = 1000000000, we get:

    X = ttb.sptenrand(sz, nonzeros=nnz)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jermyer/pyttb/pyttb/sptensor.py", line 3709, in sptenrand
    return ttb.sptensor.from_function(unit_uniform, shape, valid_nonzeros)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jermyer/pyttb/pyttb/sptensor.py", line 214, in from_function
    assert False, (
AssertionError: Requested number of nonzeros must be positive and less than the total size

Checking np.prod(size) used in the assertion, np.prod(size) = -6892481975075995648 .

Solution

Use prod from the math module to get the correct result, i.e.,

>>> from math import prod
>>> sz = (50000000,50000000,100,50000)
>>> prod(sz)
12500000000000000000000

This has the added bonus that sz is a python tuple, not a numpy object, as pointed out by @dmdunla.

@jeremy-myers jeremy-myers self-assigned this Jun 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

1 participant