Full analysis of python current path and import package path problem

What is the difference between modules, libraries and packages in python? - laike9m's answer - know
module: A .py file is a module
lib: abstract concept, not the same as the other two, as long as you like, everything is lib, even if there is only one hello world
package: It's a folder with __init__.py, it doesn't care what's in it, but generally it will contain some packages/modules

If you have the following questions, then this article is for you!

• Subquestion: Why is it normal to run unit tests in pycharm? But still there is a package import error when running in the terminal?
• Sub-question: Pycharm runs normally, but the terminal runs with an error: ModuleNotFoundError: No module named
• Sub-question: Why is the path of python running in vscode inconsistent with pycharm
• Sub-question: VSCode cannot find the relative path file

What is the current path?

Is the so-called current path the path of the input command or py script file is located?
An interpolation: Linux systems view the current path command pwd , view the current path is Python The os.getcwd ()

❓ question one 👉🏻: python the current path of the 061dfce6b22f61 program the path to execute the python script or the path that the python script says?

That is, when the following command is executed, the so-called current path is testing folder is located or the path where the main.py file is located.

python testing/main.py

✅ Answer: The current path is the path where you entered the run command, not the path where the py

For the commands below 👇, it doesn't matter to distinguish these two paths, but the paths above 👆 are different

python main.py

The relationship between the import package path and the current path?

How does knowing this knowledge help in writing programs to avoid pitfalls? , and then look down!

❓ Question 2 👉🏻: How to check the Python package path?

• Sub-question: What is the import order of python?
• Sub-question: Which folders will python go to to find packages when importing packages?
• Sub-question: Which paths will python go to to find packages when importing packages?

I /Users/bot/Desktop/code/ideaboom new one called testing folder, and testing create a new folder under main.py file.

main.py file are as follows:

import os
import sys

print('当前工作路径: ', os.getcwd())
print('导包路径为: ')

for p in sys.path:
    print(p)

and run the command /Users/bot/Desktop/code/ideaboom python testing/main.py

The program output is as follows:

当前工作路径:  /Users/bot/Desktop/code/ideaboom
导包路径为: 
/Users/bot/Desktop/code/ideaboom/testing
/Library/Frameworks/Python.framework/Versions/3.9/lib/python39.zip
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload
/Users/bot/.local/share/virtualenvs/ideaboom-8ZWsq-JB/lib/python3.9/site-packages

• Phenomenon 1 👀: We can see that there are many paths for sys.path packages. 061dfce6b23269 returns a list object. When searching for packages, it will start early from the first element of the list. For example, import django will go to /Users/bot/Desktop/code/ideaboom/testing check if there is any django package or django.py file. Then go to /Library/Frameworks/Python.framework/Versions/3.9/lib/python39.zip etc. to find them in turn.

  A package in python is the folder __init__.py

• Phenomenon 2 👀: You can see that the current path is python testing/main.py command is executed, but the path of the import package is not the path where the command is executed, but the path where the main.py
• Phenomenon 3👀: The first place in sys.path is the path where the main.py The system paths are all slightly behind.

What's wrong with the primary import package path not being the current path?

This is a very typical problem. We often build a testing folder under the root directory of the project and put the files related to unit testing.

But when we enter the command python testing/main.py ModuleNotFoundError: No module named xxx will appear. The reason for the appearance is as mentioned above: the primary package path is not the current path

Originally, the xxx and testing folders are under the root directory of the project. The primary package path in sys.path is the root directory of the project, but when we are python testing/main.py , the primary package path becomes testing instead of the project root directory. ! This is still main.py in import xxx Of course, I can't find it.

Know how to solve the problem?

Solve what 😨? Of course, it is the problem of running the tetsing file under the main.py error ModuleNotFoundError: No module named xxx . 🤯

❓ Question 3: How to change the primary package path of the Python program?

The first import package path of python is sys.path list, that is, the folder path where the py file to be run is located

Option 1: Dynamically modify sys.path

The most common way is:

Add the current path to sys.path , and to avoid naming conflicts, it is better to add it to the head of the list, rather than to the tail append As for the original ( /Users/bot/Desktop/code/ideaboom/testing ) primary package path 061dfce6b2350d, it can be deleted or retained.

import os
import sys

print('当前工作路径: ', os.getcwd())
print('导包路径为: ')

sys.path.insert(0,os.getcwd()) # 把当前路径添加到 sys.path 中

for p in sys.path:
    print(p)

Program output:👇

当前工作路径:  /Users/bot/Desktop/code/ideaboom
导包路径为: 
/Users/bot/Desktop/code/ideaboom
/Users/bot/Desktop/code/ideaboom/testing
/Library/Frameworks/Python.framework/Versions/3.9/lib/python39.zip
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload
/Users/bot/.local/share/virtualenvs/ideaboom-8ZWsq-JB/lib/python3.9/site-packages

However, this solution is not very good, and has some shortcomings, such as the following code, which looks very inelegant, because according to the python code specification, the code related to the import package should be written at the top. This package guide + code + guide The way the package pythonic

import os
import sys
import time
import schedule
from pathlib import Path
import os
import sys
import time
import schedule
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent.parent
sys.path.insert(0, str(BASE_DIR))

import django
django.setup()

Option 2: Use the environment variable PYTHONPATH

🥳 Better solution👇

Because the setting of the primary package path is the default execution of the python interpreter, can we specify the primary package path we need before the python interpreter starts?

By looking at the python --help command, we can get to the following:👇

usage: /opt/homebrew/Cellar/python@3.8/3.8.12/bin/python3 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
-b     : issue warnings about str(bytes_instance), str(bytearray_instance)
         and comparing bytes/bytearray with str. (-bb: issue errors)
-B     : don't write .pyc files on import; also PYTHONDONTWRITEBYTECODE=x
-c cmd : program passed in as string (terminates option list)
-d     : debug output from parser; also PYTHONDEBUG=x
-E     : ignore PYTHON* environment variables (such as PYTHONPATH)
-h     : print this help message and exit (also --help)
-i     : inspect interactively after running script; forces a prompt even
         if stdin does not appear to be a terminal; also PYTHONINSPECT=x
-I     : isolate Python from the user's environment (implies -E and -s)
-m mod : run library module as a script (terminates option list)
-O     : remove assert and __debug__-dependent statements; add .opt-1 before
         .pyc extension; also PYTHONOPTIMIZE=x
-OO    : do -O changes and also discard docstrings; add .opt-2 before
         .pyc extension
-q     : don't print version and copyright messages on interactive startup
-s     : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S     : don't imply 'import site' on initialization
-u     : force the stdout and stderr streams to be unbuffered;
         this option has no effect on stdin; also PYTHONUNBUFFERED=x
-v     : verbose (trace import statements); also PYTHONVERBOSE=x
         can be supplied multiple times to increase verbosity
-V     : print the Python version number and exit (also --version)
         when given twice, print more information about the build
-W arg : warning control; arg is action:message:category:module:lineno
         also PYTHONWARNINGS=arg
-x     : skip first line of source, allowing use of non-Unix forms of #!cmd
-X opt : set implementation-specific option. The following options are available:

         -X faulthandler: enable faulthandler
         -X showrefcount: output the total reference count and number of used
             memory blocks when the program finishes or after each statement in the
             interactive interpreter. This only works on debug builds
         -X tracemalloc: start tracing Python memory allocations using the
             tracemalloc module. By default, only the most recent frame is stored in a
             traceback of a trace. Use -X tracemalloc=NFRAME to start tracing with a
             traceback limit of NFRAME frames
         -X showalloccount: output the total count of allocated objects for each
             type when the program finishes. This only works when Python was built with
             COUNT_ALLOCS defined
         -X importtime: show how long each import takes. It shows module name,
             cumulative time (including nested imports) and self time (excluding
             nested imports). Note that its output may be broken in multi-threaded
             application. Typical usage is python3 -X importtime -c 'import asyncio'
         -X dev: enable CPython's "development mode", introducing additional runtime
             checks which are too expensive to be enabled by default. Effect of the
             developer mode:
                * Add default warning filter, as -W default
                * Install debug hooks on memory allocators: see the PyMem_SetupDebugHooks() C function
                * Enable the faulthandler module to dump the Python traceback on a crash
                * Enable asyncio debug mode
                * Set the dev_mode attribute of sys.flags to True
                * io.IOBase destructor logs close() exceptions
         -X utf8: enable UTF-8 mode for operating system interfaces, overriding the default
             locale-aware mode. -X utf8=0 explicitly disables UTF-8 mode (even when it would
             otherwise activate automatically)
         -X pycache_prefix=PATH: enable writing .pyc files to a parallel tree rooted at the
             given directory instead of to the code tree

--check-hash-based-pycs always|default|never:
    control how Python invalidates hash-based .pyc files
file   : program read from script file
-      : program read from stdin (default; interactive mode if a tty)
arg ...: arguments passed to program in sys.argv[1:]

Other environment variables:
PYTHONSTARTUP: file executed on interactive startup (no default)
PYTHONPATH   : ':'-separated list of directories prefixed to the
               default module search path.  The result is sys.path.
PYTHONHOME   : alternate <prefix> directory (or <prefix>:<exec_prefix>).
               The default module search path uses <prefix>/lib/pythonX.X.
PYTHONCASEOK : ignore case in 'import' statements (Windows).
PYTHONUTF8: if set to 1, enable the UTF-8 mode.
PYTHONIOENCODING: Encoding[:errors] used for stdin/stdout/stderr.
PYTHONFAULTHANDLER: dump the Python traceback on fatal errors.
PYTHONHASHSEED: if this variable is set to 'random', a random value is used
   to seed the hashes of str and bytes objects.  It can also be set to an
   integer in the range [0,4294967295] to get hash values with a
   predictable seed.
PYTHONMALLOC: set the Python memory allocators and/or install debug hooks
   on Python memory allocators. Use PYTHONMALLOC=debug to install debug
   hooks.
PYTHONCOERCECLOCALE: if this variable is set to 0, it disables the locale
   coercion behavior. Use PYTHONCOERCECLOCALE=warn to request display of
   locale coercion and locale compatibility warnings on stderr.
PYTHONBREAKPOINT: if this variable is set to 0, it disables the default
   debugger. It can be set to the callable of your debugger of choice.
PYTHONDEVMODE: enable the development mode.
PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files.

After looking around, I feel that this PYTHONHOME is the savior, but in fact it is not, it is precisely the inconspicuous PYTHONPATH

testing/main.py

import os
import sys

print('当前工作路径: ', os.getcwd())
print('导包路径为: ')

for p in sys.path:
    print(p)

We can start the program in the following ways:👇

PYTHONPATH=$(pwd)  python testing/main.py

At this point the output of the program becomes: 👇

(ideaboom) ╭─[e-mail] ~/Desktop/code/ideaboom  ‹master*› 
╰─➤  PYTHONPATH=$(pwd)  python testing/main.py
当前工作路径:  /Users/bot/Desktop/code/ideaboom
导包路径为: 
/Users/bot/Desktop/code/ideaboom/testing
/Users/bot/Desktop/code/ideaboom
/Library/Frameworks/Python.framework/Versions/3.9/lib/python39.zip
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload
/Users/bot/.local/share/virtualenvs/ideaboom-8ZWsq-JB/lib/python3.9/site-packages

Let's take another look at PYTHONPATH=$(pwd) python testing/main.py , which is equivalent to PYTHONPATH=/Users/bot/Desktop/code/ideaboom python testing/main.py .

python testing/main.py added to PYTHONPATH=$(pwd) is limited to the execution of this command, and will not spread to the current shell environment.