VMware ESXi and vSphere Cluster Management

Find Files and Understand Search Paths in Python

Learn the difference between finding files on disk and resolving Python imports with pathlib, sys.path, PYTHONPATH, working directories, and safe diagnostics.

Python uses several different ideas of a path. A filesystem path identifies a file or directory. An import path tells Python where to look for modules and packages. These systems overlap, but they are not the same.

This distinction explains many common errors: a file can exist on disk while still being unavailable to import, and a relative filename can work in one launch context but fail in another.

Filesystem searching versus module searching

To locate an ordinary file such as report.csv, search the filesystem with tools such as pathlib, glob, os.walk(), or os.scandir(). These tools inspect directories and filenames.

To resolve an import such as import requests, Python follows import rules and consults the ordered locations in sys.path. sys.path is not a general recursive disk search. Python does not automatically scan every directory below every drive.

Inspect Python's import search path

A module is an importable Python source file or extension module. A package is a structured collection of modules, usually represented by a directory. Python searches the locations in sys.path in order and uses the first suitable match it finds.

import sys

for location in sys.path:
    print(location)

Entries can represent directories and, in some configurations, archive locations. The exact list varies with the operating system, Python installation, virtual environment, launch method, IDE, notebook, test runner, and project configuration.

Search order matters. Suppose both project_a/helpers.py and project_b/helpers.py are available. If project_a occurs first in sys.path, import helpers normally loads the module from project_a. This unintended selection is called module shadowing.

Where import locations come from

Python's import locations commonly include the following:

• The current working directory or a script-related startup location. The exact first entry depends on whether Python was launched with a script, the -m option, -c, an interactive shell, or another tool.
• Directories named by the PYTHONPATH environment variable.
• Standard-library directories belonging to the active Python installation.
• site-packages, a common directory for installed third-party packages.
• Directories associated with the active virtual environment.
• Additional locations influenced by site configuration and .pth files. These mechanisms can add import locations during Python startup.

Inspect the current working directory

The current working directory is the directory used as the base for relative filesystem paths by the running process. It is not necessarily the directory containing your Python source file.

import os
from pathlib import Path

print(os.getcwd())
print(Path.cwd())

For example, Path('data/input.csv') means “the data/input.csv path below the current working directory.” An editor, terminal, notebook, test runner, service, or deployment system may choose a different working directory when it starts your program.

Changing the working directory

Use os.chdir() when a process genuinely needs to operate from another directory. It accepts a string or a path-like object such as Path.

import os
from pathlib import Path

target = Path.home() / 'example-project'
os.chdir(target)
print('Now running from:', Path.cwd())

On Windows, use Path, a raw string, escaped backslashes, or forward slashes:

from pathlib import Path

windows_path = Path(r'C:\Users\Sam\example-project')
# Equivalent style:
other_path = Path('C:/Users/Sam/example-project')

Changing the working directory affects the entire running process. Code that changes it can surprise other functions and libraries, so prefer explicit absolute paths or paths based on a known project location when practical.

Temporarily adding an import location

You can modify sys.path for the current Python process. append() adds a directory after existing entries; insert(0, ...) deliberately gives the directory high priority.

import sys
from pathlib import Path

development_dir = Path('/work/my-project')

# Search it after the existing locations.
sys.path.append(str(development_dir))

# Or intentionally prioritize it:
# sys.path.insert(0, str(development_dir))

import mymodule

These changes normally disappear when the process exits. Inserting at position zero can also cause module shadowing: a local file with a familiar name might replace the standard-library or installed module you intended to use. Inspect the result rather than assuming the import succeeded from the expected location.

Persistent import path configuration

PYTHONPATH adds directories to Python's import search path when Python starts. The path separator is usually a colon on POSIX shells and a semicolon on Windows.

# POSIX shell: current shell session
export PYTHONPATH='/work/shared-libraries:/work/my-project'
python app.py

# Windows Command Prompt: current command prompt session
set PYTHONPATH=C:\work\shared-libraries;C:\work\my-project
python app.py

# PowerShell: current PowerShell session
$env:PYTHONPATH = 'C:\work\shared-libraries;C:\work\my-project'
python app.py

For repeatable projects, a per-project virtual environment, properly installed packages, project packaging, or an editable install is usually safer than globally modifying PYTHONPATH. Persistent environment changes can affect unrelated programs and make the active configuration difficult to reproduce.

Portable filesystem paths with pathlib

pathlib.Path is Python's object-oriented, cross-platform interface for filesystem paths. Use the / operator or joinpath() instead of manually concatenating strings.

from pathlib import Path

project = Path.cwd()
config_file = project / 'config' / 'settings.json'
logs_dir = project.joinpath('var', 'logs')

print(config_file)
print(logs_dir)

Finding ordinary files on disk

Test a known path

from pathlib import Path

candidate = Path('data/report.csv')
print('Exists:', candidate.exists())
print('Is file:', candidate.is_file())
print('Is directory:', candidate.is_dir())

exists() checks whether the path exists, while is_file() and is_dir() distinguish common filesystem object types. A relative candidate is interpreted from Path.cwd(); an absolute path is independent of the working directory.

Match files in a directory

from pathlib import Path

reports_dir = Path('reports')
for path in reports_dir.glob('*.csv'):
    if path.is_file():
        print(path)

glob() performs pattern-based filename matching, such as *.csv. You can filter by extension, name, directory, or type:

for path in Path('project').glob('*'):
    if path.is_file() and path.suffix.lower() in {'.jpg', '.png'}:
        print(path)

Search recursively

from pathlib import Path

root = Path('project')
for path in root.rglob('*.csv'):
    if path.is_file():
        print(path)

rglob() descends through subdirectories. For detailed traversal control, use os.walk():

import os

for directory, subdirectories, filenames in os.walk('project'):
    for filename in filenames:
        if filename.endswith('.csv'):
            print(os.path.join(directory, filename))

os.scandir() is useful when examining one directory and needing directory-entry metadata efficiently:

import os

with os.scandir('project') as entries:
    for entry in entries:
        if entry.is_file() and entry.name.endswith('.txt'):
            print(entry.path)

Verify import resolution

When debugging imports, print the relevant paths instead of relying on a directory listing alone. Package names, package structure, and search precedence all affect resolution.

import importlib.util
import sys
from pathlib import Path

print('Working directory:', Path.cwd())
print('Import locations:')
for location in sys.path:
    print(' ', location)

spec = importlib.util.find_spec('json')
print('json spec:', spec)
print('json origin:', spec.origin if spec else None)

find_spec() checks whether Python can resolve a module and returns a specification containing information such as its origin. It does not normally load the target module itself, although resolving a submodule can involve its parent package.

After importing a module, module.__file__ often identifies the file that was loaded:

import json

print(json.__file__)

Troubleshooting common path problems

“ModuleNotFoundError” even though the file exists

• Print sys.path and confirm that the directory containing the package is present.
• Check that the active interpreter is the one associated with the intended virtual environment.
• Put the package's containing directory on the path when appropriate, rather than blindly adding the package directory itself.
• Use importlib.util.find_spec('package_name') to inspect resolution.
• For a lasting solution, use a virtual environment and install the project or package correctly. Use a temporary sys.path change only for controlled, short-lived situations.

A relative file works in one launch context but not another

• Print Path.cwd().
• Compare the working directory used by the terminal, IDE, notebook, test runner, or service.
• Use an explicit documented base directory or an absolute path, or configure the launch working directory.

A Windows path is invalid or raises an escape error

• Inspect backslashes in the string literal.
• Prefer Path(r'C:\folder\file.txt'), escaped backslashes, or Path('C:/folder/file.txt').

Python imports the wrong module

• Print module.__file__ after importing.
• Look for a same-named local file or package.
• Review the order of sys.path and remove unnecessary priority insertions.
• Rename conflicting local files and use an isolated environment with a clear project structure.

Recursive searching is slow or fails

• Choose a narrower search root and filename pattern.
• Identify unreadable directories and decide how permission exceptions should be handled.
• Consider symbolic-link behavior and avoid traversing locations that are not part of the project.

Exam-relevant distinctions

• Relative path: interpreted from the current working directory.
• Absolute path: identifies a location independently of the current working directory.
• sys.path: an ordered list used for import resolution, not recursive ordinary-file searching.
• PYTHONPATH: an environment variable that can add import locations at startup.
• append() versus insert(0, ...): both alter the current process, but insertion at zero gives the new location priority.
• Module shadowing: an unintended same-named module wins because it appears earlier in the import search path.
• glob() versus rglob(): rglob() recursively descends into subdirectories.

Practical checklist

1. Decide whether you are locating an ordinary filesystem object or resolving an import.
2. For files, inspect Path.cwd(), construct paths with Path, and use exists(), glob(), or rglob().
3. For imports, print sys.path and check the active interpreter or virtual environment.
4. Use find_spec() and __file__ to verify what Python will resolve or has loaded.
5. Prefer project packaging, editable installs, and virtual environments over permanent global path hacks.