Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pylint does not follow python import order : Builtin modules are imported before custom modules #6535

Open
electriquo opened this issue May 7, 2022 · 8 comments · May be fixed by pylint-dev/astroid#2223
Labels
False Positive 🦟 A message is emitted but nothing is wrong with the code Import system Needs astroid update Needs an astroid update (probably a release too) before being mergable Needs PR This issue is accepted, sufficiently specified and now needs an implementation python 3.10

Comments

@electriquo
Copy link

electriquo commented May 7, 2022

Bug description

when a project holds a module name that shadows builtin module name, pylist fails on no-member, when clearly this is not the case.

reproduction

given 2 files:

  1. os.py an empty file
  2. test.py with the following content
    """test.py"""
    
    import os
    print(os.environ['SHELL'])

and now pylint fails on no-member as follows

$ pylint **/*.py
************* Module test
test.py:4:6: E1101: Module 'os' has no 'environ' member (no-member)

------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)

but the code is correct

$ python test.py
/usr/local/bin/bash

Configuration

No response

Command used

pylint **/*.py

Pylint output

************* Module test
test.py:4:6: E1101: Module 'os' has no 'environ' member (no-member)

------------------------------------------------------------------
Your code has been rated at 0.00/10 (previous run: 0.00/10, +0.00)

Expected behavior

pylint should be aware of the import tree, and report such issue if that is indeed the case.

in the reproduction snippet above, test.py imports the os builtin module (and not os.py) that has the environ().

Pylint version

pylint 2.13.8
astroid 2.11.4
Python 3.10.3 (main, Mar 17 2022, 09:46:58) [Clang 13.1.6 (clang-1316.0.21.2)]

OS / Environment

macos

$ sw_vers
ProductName:	macOS
ProductVersion:	12.3.1
BuildVersion:	21E258

Additional dependencies

No response

@electriquo electriquo added the Needs triage 📥 Just created, needs acknowledgment, triage, and proper labelling label May 7, 2022
@electriquo electriquo changed the title "no-member" error "no-member" false-positive error May 7, 2022
@electriquo electriquo changed the title "no-member" false-positive error "no-member" false-positive failure May 7, 2022
@Pierre-Sassoulas Pierre-Sassoulas added Bug 🪲 Import system False Positive 🦟 A message is emitted but nothing is wrong with the code and removed Needs triage 📥 Just created, needs acknowledgment, triage, and proper labelling labels May 7, 2022
@mbyrnepr2
Copy link
Member

In my case I haven't been able to reproduce the no-member. I see an issue for Python in this situation also:

Create os.py in the current directory and run python test.py

(venv39) programming python test.py
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site.py", line 589, in <module>
    main()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site.py", line 566, in main
    known_paths = removeduppaths()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site.py", line 127, in removeduppaths
    dir, dircase = makepath(dir)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site.py", line 92, in makepath
    dir = os.path.join(*paths)
AttributeError: module 'os' has no attribute 'path'

@Pierre-Sassoulas Pierre-Sassoulas added Needs PR This issue is accepted, sufficiently specified and now needs an implementation and removed Bug 🪲 labels Jul 18, 2022
@Pierre-Sassoulas Pierre-Sassoulas changed the title "no-member" false-positive failure Pylint do not follow python import order : Builtin modules are imported before custom modules Jul 18, 2022
@Pierre-Sassoulas
Copy link
Member

I can reproduce this, I wonder if the order is interpreter dependent. If that's the case we need to handle this dynamically.

@ssbarnea
Copy link
Contributor

ssbarnea commented Jul 18, 2022

I encountered this issue multiple times over the last month and that is because my main python is 3.11 and pylint is not yet compatible with 3.11.

So, I forced us to use oldest supported python and lock it during linting. I hope to see py311 support added soon in astroid.

ssbarnea added a commit to ssbarnea/molecule-podman that referenced this issue Jul 18, 2022
ssbarnea added a commit to ansible-community/molecule-podman that referenced this issue Jul 19, 2022
@zifter
Copy link

zifter commented Nov 1, 2022

I have the same issue.
I'm using to unit test library for multiple python versions - from 3.6 to 3.10.
pylint 2.13.9 is used to check
And now I'd like to test on 3.11, so I have upgraded unit tests to use python 3.11 (alongside with other versions) and only on py3.11 something when wrong and pylint gives me false positive results:

************* Module clickhouse_migrations.cmd
src/clickhouse_migrations/cmd.py:27:16: E1101: Module 'os' has no 'environ' member (no-member)
src/clickhouse_migrations/cmd.py:32:16: E1101: Module 'os' has no 'environ' member (no-member)
src/clickhouse_migrations/cmd.py:37:16: E1101: Module 'os' has no 'environ' member (no-member)
src/clickhouse_migrations/cmd.py:42:16: E1101: Module 'os' has no 'environ' member (no-member)
src/clickhouse_migrations/cmd.py:[47](https://github.com/zifter/clickhouse-migrations/actions/runs/3338374031/jobs/5596408168#step:6:48):16: E1101: Module 'os' has no 'environ' member (no-member)
src/clickhouse_migrations/cmd.py:53:16: E1101: Module 'os' has no 'environ' member (no-member)
************* Module clickhouse_migrations.types
src/clickhouse_migrations/types.py:16:17: E1101: Module 'os' has no 'scandir' member (no-member)
************* Module clickhouse_migrations.defaults
src/clickhouse_migrations/defaults.py:2:0: E0611: No name 'path' in module 'os' (no-name-in-module)
src/clickhouse_migrations/defaults.py:9:30: E1101: Module 'os' has no 'getcwd' member (no-member)

I hope this issues will be resolved.

@chuma9615
Copy link

I encountered a similar error that can shed light to what is going on

Python version 3.11
Pylint version 2.15.8

We have a module in our source code os_sensors.py that gets confused with the os standard module and for every file that the os standard module is imported, pylint throws this kind of errors

************* Module components.client
components/client.py:126:37: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:131:23: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:132:35: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:175:19: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:176:31: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:178:15: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:208:19: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:209:30: E1101: Module 'os' has no 'path' member (no-member)
components/client.py:382:11: E1101: Module 'os' has no 'path' member (no-member)
************* Module components.file_processor
components/file_processor.py:103:[28](https://github.com/Blissway/wale_device/actions/runs/3651051551/jobs/6167801259#step:7:29): E1101: Module 'os' has no 'listdir' member (no-member)
components/file_processor.py:1[32](https://github.com/Blissway/wale_device/actions/runs/3651051551/jobs/6167801259#step:7:33):12: E1101: Module 'os' has no 'replace' member (no-member)

A quick solution was to change from os_sensors.py to system_sensors.py and it worked!

I'm guessing it's a regex that is wrongly matching imports or something like that.

@Pierre-Sassoulas Pierre-Sassoulas pinned this issue Dec 11, 2022
@pierrecdn
Copy link

I understand most projects linking issues here are postponing their 3.11 support due to this.
In my case I observed this while having no os*.py file in my project (I may have some through transitive dependencies though).

Anyway, if anybody following this have more info, I'd be interested.

@DanielNoord
Copy link
Collaborator

According to the OP this isn't 3.11 related but was also present on 3.10. Perhaps you're experiencing a different issue.

@jacobtylerwalls jacobtylerwalls added Needs astroid update Needs an astroid update (probably a release too) before being mergable and removed Help wanted 🙏 Outside help would be appreciated, good for new contributors labels Jun 23, 2023
jacobtylerwalls added a commit to jacobtylerwalls/astroid that referenced this issue Jun 23, 2023
For example:
`import copy` now finds `copy` instead of `copy.py`.

This worked correctly before in at least some cases if there
was a (more?) complete chain of __init__.py files from cwd
all the way to the location of the `copy.py` module.

Closes pylint-dev/pylint#6535
@jacobtylerwalls jacobtylerwalls modified the milestones: 2.17.6, 3.0.0a7 Jun 23, 2023
@jacobtylerwalls
Copy link
Member

jacobtylerwalls commented Jun 23, 2023

I would be grateful for anyone's input on pylint-dev/astroid#2223, which should fix the OP.

By installing from that branch and retesting your use case, you can give us the confidence to make changes to a delicate part of this tool's system. Thanks!

jacobtylerwalls added a commit to jacobtylerwalls/astroid that referenced this issue Jun 26, 2023
For example:
`import copy` now finds `copy` instead of `copy.py`.

This worked correctly before in at least some cases if there
was a (more?) complete chain of __init__.py files from cwd
all the way to the location of the `copy.py` module.

Closes pylint-dev/pylint#6535
jacobtylerwalls added a commit to jacobtylerwalls/astroid that referenced this issue Jun 26, 2023
For example:
`import copy` now finds `copy` instead of `copy.py`.

This worked correctly before in at least some cases if there
was a (more?) complete chain of __init__.py files from cwd
all the way to the location of the `copy.py` module.

Closes pylint-dev/pylint#6535
@jacobtylerwalls jacobtylerwalls modified the milestones: 3.0.0a7, 3.0.0b1 Jul 4, 2023
@jacobtylerwalls jacobtylerwalls removed this from the 3.0.0b1 milestone Jul 30, 2023
@jacobtylerwalls jacobtylerwalls removed their assignment Jul 30, 2023
darcythomas pushed a commit to darcythomas/graphviz that referenced this issue Sep 4, 2023
Ubuntu 22.10 was EOLed on 2023-07-26.¹

The obvious thing would be to _upgrade_ to Ubuntu 23.04. But we run into the new
Python policy Ubuntu has instituted (see
0050eac). We could comply with this or move
this task to Fedora, but then we find that Pylint does not work with Python
3.11² which is what is in use on both Ubuntu 23.04 and all supported Fedora
releases. So downgrading to Ubuntu 22.04 seems like the simplest options for an
environment that is still in its support window.

¹ https://wiki.ubuntu.com/Releases
² Discussed in the comments of
  pylint-dev/pylint#6535. Symptoms are complaints of
  `os` members not existing.
@jacobtylerwalls jacobtylerwalls changed the title Pylint do not follow python import order : Builtin modules are imported before custom modules Pylint does not follow python import order : Builtin modules are imported before custom modules Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
False Positive 🦟 A message is emitted but nothing is wrong with the code Import system Needs astroid update Needs an astroid update (probably a release too) before being mergable Needs PR This issue is accepted, sufficiently specified and now needs an implementation python 3.10
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants