pants lint :: and pants lint /some/path behave differently with ruff, but not isort for sorting python deps

[benceferdinandy-signifyd]

Describe the bug

We're trying to switch from using black and isort to ruff. I'm aware of this bug, but I don't think it's this one, because it does not trigger using isort only with ruff. For context, our monorepo has multiple resolves and several packages in each resolve, which might affect this.

My starting state is isort configured for sorting python import.

Running pants lint :: is green, and running pants lint /path/to/a/specific/file.py is also green.

After removing isort from linting and adding ruff the behaviour changes:

Running pants lint :: is showing a bunch of import sorting errors, including in /path/to/a/specific/file.py, but running pants lint /path/to/a/specific/file.py does NOT so an error.

Running ruff directly (uv run ruff check /path/to/a/specific/file.py (version 0.11.5)) also shows the error.

This is especially problematic, because pants fix :: behaves as the individual file linting (so does pants fix /path/to/a/specific/file.py), which means it is not possible to achieve a state where pants lint :: and pants fix :: agree, if I manually format files by running ruff directly pants fix will change it back. Note that running ruff directly on the entire repo, or just a single file will always fix the import order to what pants lint :: is expecting.

Pants version
We're using 2.28, but I can reproduce with 2.31.0.dev1 as well.

OS
MacOS (I have not tested on linux)

Additional info
Add any other information about the problem here, such as attachments or links to gists, if relevant.

[benjyw]

Hmm, how annoying. I would guess that this is a similar issue to #15069, where Ruff infers import groupings (first party vs third party) by examining the files presented to it (e.g., as described in https://docs.astral.sh/ruff/settings/#lint_isort_detect-same-package), so that inference can change depending on what's in the sandbox.

Do the sorting errors you see involve things moving from first-party to third-party grouping or vice versa?

In that case a workaround might be to specify what your first-party packages are explicitly, using
https://docs.astral.sh/ruff/settings/#lint_isort_known-first-party
and/or
https://docs.astral.sh/ruff/settings/#lint_isort_known-third-party

[benjyw]

Probably in the future we should stop even trying to restrict the files we put in front of ruff. It is so fast that there is likely no benefit to not letting it just run on everything every time...

[benceferdinandy-signifyd]

Hmm, how annoying. I would guess that this is a similar issue to #15069, where Ruff infers import groupings (first party vs third party) by examining the files presented to it (e.g., as described in https://docs.astral.sh/ruff/settings/#lint_isort_detect-same-package), so that inference can change depending on what's in the sandbox.

Do the sorting errors you see involve things moving from first-party to third-party grouping or vice versa?

Yes, although since my understanding of the lingo related to this is a bit poor, I'm going to give the exact structure here:

The file in question is projects/fire/fire_infrastructure/fire_infrastructure/widgets/plots.py this is the change that is either being made or not:

--- a/projects/fire/fire_infrastructure/fire_infrastructure/widgets/plots.py
+++ b/projects/fire/fire_infrastructure/fire_infrastructure/widgets/plots.py
@@ -1,12 +1,13 @@
 from copy import deepcopy
 
-import fire_infrastructure.constants.fields as fd
 import pandas as pd
 import plotly.graph_objects as go
 from IPython.display import Markdown, display
 from ipywidgets import widgets
 from plotly.subplots import make_subplots
 
+import fire_infrastructure.constants.fields as fd
+ 

projects/fire/fire_infrastructure has the BUILD file for pants and requirements.txt

I read the issue you mentioned before posting this, and what baffles me, is that I would expect that our original setup with isort should behave the same, but it does not.

Our original isort config is

[tool.isort]
profile = "black"
line_length = 140
color_output = true

Grepping for isort in the repo does show a few isort:skip comments, but the specific file I'm testing doesn't have it. Although it might just be a difference in how isort works, because running isort on the file outside of pants doesn't format it either, so that one is consistent.

In that case a workaround might be to specify what your first-party packages are explicitly, using https://docs.astral.sh/ruff/settings/#lint_isort_known-first-party and/or https://docs.astral.sh/ruff/settings/#lint_isort_known-third-party

I haven't quite yet figured out how to do this, without requiring people to update pyproject.toml every time we add a new package.

[benceferdinandy-signifyd]

Probably in the future we should stop even trying to restrict the files we put in front of ruff. It is so fast that there is likely no benefit to not letting it just run on everything every time...

Yeah, it's probably easier not to try to do anything here and just let ruff handle it, so it's surely consistent always. We have 6600 python files in our repository and a cold run is 0.48s on my machine, so not really a bottleneck to anything in practice.

[benceferdinandy-signifyd]

Unfortunately, globbing in known-third-party won't work, since of course, the files are not there and we don't have a naming pattern like prefix companyi name before package. So afaik we either need to leave isorting to isort (as that seems to work) or have somehow pants generate the module names and pass it to ruff via cli arg during runtime.

[benjyw]

In that case a workaround might be to specify what your first-party packages are explicitly, using https://docs.astral.sh/ruff/settings/#lint_isort_known-first-party and/or https://docs.astral.sh/ruff/settings/#lint_isort_known-third-party

I haven't quite yet figured out how to do this, without requiring people to update pyproject.toml every time we add a new package.

So your first-party packages aren't under some single top-level name, like your company name or something?

[benceferdinandy-signifyd]

In that case a workaround might be to specify what your first-party packages are explicitly, using https://docs.astral.sh/ruff/settings/#lint_isort_known-first-party and/or https://docs.astral.sh/ruff/settings/#lint_isort_known-third-party

I haven't quite yet figured out how to do this, without requiring people to update pyproject.toml every time we add a new package.

So your first-party packages aren't under some single top-level name, like your company name or something?

Yeah, unfortunately, no. I vibecoded a PoC that overrides how pants call ruff by injecting dynamically a config option for known-first-party by parsing the BUILD files. It not only sounds but looks hacky, but works in the sense that now all variations of lint/fix calls are consistent. I'm now scratching my head over the fact, that the following methods for isorting find successively more first-parties:

• running ruff directly without pants on the entire repo
• pants lint :: without my hack
• pants lint :: with first-parties sourced directly from BUILD files

The last two could be because of resolves maybe?

[benceferdinandy-signifyd]

Just for posterity, we decided to work around this by setting for now:

[tool.ruff.lint.isort]
no-sections = true

Turns out the github UI sort of breaks on PR-s with more than 1000 files :)