Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Different types of imports get disconnected. #224

Open
luketych opened this issue May 30, 2024 · 3 comments
Open

Different types of imports get disconnected. #224

luketych opened this issue May 30, 2024 · 3 comments

Comments

@luketych
Copy link

I have a project that combines two different import methods:

import a from b

and import c

I am just making up this example, and will show more details if necessary.

I am using pydeps . --include-missing

Here is the generated graph.

Screenshot 2024-05-29 at 8 29 13 PM
@thebjorn
Copy link
Owner

I'm not sure what you are asking...? Generally, you'll need to

  1. show what you are doing
  2. show what is happening
  3. describe what you expected to happen instead
  4. if you think it is a bug, you will also need to provide the smallest possible sample that I can run that demonstrates the problem (e.g. it can't contain source that I cannot access).

1-3 lets me know if your understanding and usage is reasonable, and 4 makes it possible for me to debug the issue and provide a solution.

@luketych
Copy link
Author

luketych commented Jun 1, 2024

It's a bit of a weird example, but I am trying to demonstrate that I can use pydeps to detect circular imports. I am expecting the generated diagram to show the circular dependency, where func1 and func2 would be caught in a circle attempting to import each other via the importer.

I am mixing importing directly, ie

➜  src git:(develop) ✗ ls

__init__.py __pycache__ func1.py    func2.py    importer.py index.py 
 ➜  src git:(develop) ✗ cat importer.py

def importer(module, function):
    module = __import__(module)
    return module.__dict__[function]
➜  src git:(develop) ✗ cat index.py

import importer

func1 = importer.importer('func1', 'func1')

if __name__ == '__main__':
    func1()

➜  src git:(develop) ✗ cat func1.py

import importer

func2 = importer.importer('func2', 'func2')

def func1():
    print("func1 in func1.py")

    func2()

➜  src git:(develop) ✗ cat func2.py

import importer

func1 = importer.importer('func1', 'func1')

def func2():
    print("func2() in func2.py")

    func1()

@thebjorn
Copy link
Owner

thebjorn commented Jun 2, 2024

If we look at your importer function:

def importer(module, function):
    module = __import__(module)
    return module.__dict__[function]

and display the corresponding byte code:

>>> import importer
>>> import dis
>>> dis.dis(importer.importer)
  2           0 LOAD_GLOBAL              0 (__import__)
              2 LOAD_FAST                0 (module)
              4 CALL_FUNCTION            1
              6 STORE_FAST               0 (module)

  3           8 LOAD_FAST                0 (module)
             10 LOAD_ATTR                1 (__dict__)
             12 LOAD_FAST                1 (function)
             14 BINARY_SUBSCR
             16 RETURN_VALUE
>>>

module = __import__(module) correspond to instructions labeled 0-6, and return module.__dict__[function] are the byte codes labeled 8-16. There are no concrete module and function names present, just the parameter names of the importer function.

pydeps doesn't run your program, so it has no idea what the function parameter will be bound to at runtime. This kind of dynamic imports makes it impossible for any static analyzer to figure out what the actual import tree looks like.

Yes, it would be theoretically possible, e.g. using dataflow analysis and whole-program analysis, but pydeps (and the Python modulefinder module) use a very simple (but fast) technique for finding imports.

If you look at the bytecode for a "normal" import:

>>> def foo():
...     import math
...
>>> dis.dis(foo)
  2           0 LOAD_CONST               1 (0)
              2 LOAD_CONST               0 (None)
              4 IMPORT_NAME              0 (math)
              6 STORE_FAST               0 (math)
              8 LOAD_CONST               0 (None)
             10 RETURN_VALUE
>>>

You'll see the IMPORT_NAME instruction, and its parameter math. When seeing this pydeps knows that the current module imports a module named math.

Pydeps can unfortunately, but by design, not follow dynamic imports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants