viable/strict/1780614985: Build torch._C extension module and version.py via CMake (#180243)

Move the build of the torch._C Python extension and the install of

torch/version.py from setup.py into torch/CMakeLists.txt, so

both the existing setuptools backend and the upcoming

scikit-build-core backend (#180247) go through the same code path.

The new Python_add_library(_C MODULE WITH_SOABI ...) target builds

the same torch/csrc/stub.c and installs at the wheel root (mapped to

torch/ by wheel.install-dir); setup.py's Extension("torch._C", ...) block and its companion extra_compile_args /

extra_link_args / make_relative_rpath_args plumbing are removed.

version.py is gitignored, so it is installed explicitly via

install(FILES ... DESTINATION ".") rather than relying on Python

package collection.

Adds a BinaryDistribution class in setup.py that forces

has_ext_modules=True so bdist_wheel uses the binary layout

(package_data + data_files only) instead of the purelib layout, which

would otherwise dump CMake's build/lib/ tree -- including test

artifacts -- into the wheel root. package_data collects the

CMake-built extension via the interpreter's exact module name

(f"_C{sysconfig.get_config_var('EXT_SUFFIX')}") rather than a _C*

glob, so a build/ dir reused across Python versions cannot leak

other ABIs' _C modules into the wheel.

Windows handling: the manual setup.py copy of _C.lib from

build_temp to build_lib is replaced by the _C target's

ARCHIVE DESTINATION "lib". For MSVC, the linker is told where to

find pythonXX.lib via /LIBPATH (Python headers use #pragma comment(lib, "pythonXX") which expects a bare-name lookup); for

Windows venvs, FindPython's library search path is extended with

sys.base_prefix/libs since the import library lives in the base

install, not the venv. nnapi_backend gains

TORCH_PYTHON_LINK_FLAGS so it picks up the same MSVC linker

options as torch_python.

Test plan

Verification is via artifact-content comparison against the parent

(#180242). Expectation: the wheel must be content-equivalent modulo

the one file the PR rebuilds (torch/_C.cpython-*.so), which must

remain functionally identical (same exports, same imports, same

rpath).

• 
  

  No regression in artifact sizes across every manywheel /
  
  libtorch artifact (sub-0.001 % to 0.001 % delta, zip-timestamp
  
  noise consistent across artifacts this PR can't directly affect):

  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  

  artifact	parent (#180242)	this PR (#180243)	delta
  manywheel-py3_12-rocm7_2	6,153,280,701	6,153,275,144	-5,557 (-0.000%)
  manywheel-py3_12-rocm7_1	5,788,503,789	5,788,499,957	-3,832 (-0.000%)
  manywheel-py3_12-cuda12_6	841,797,521	841,797,813	+292 (+0.000%)
  manywheel-py3_12-cuda13_2	624,213,880	624,212,127	-1,753 (-0.000%)
  manywheel-py3_12-cpu	189,248,452	189,248,117	-335 (-0.000%)
  manywheel-py3_12-xpu	778,712,445	778,711,285	-1,160 (-0.000%)
  libtorch-rocm7_2-shared-with-deps-release	12,117,315,856	12,117,314,539	-1,317 (-0.000%)
  libtorch-cpu-shared-with-deps-release	257,626,615	257,624,612	-2,003 (-0.001%)

  
  


• 
  

  Wheel manifest equivalence vs parent
  
  (manywheel-py3_12-rocm7_2): both wheels contain 26,779 entries;
  
  filename diff is empty -- no files added or removed. Confirms the
  
  BinaryDistribution layout fix worked (no build/lib/ spillage
  
  into the wheel root).

  
  


• 
  

  Per-file size diff: exactly one file changed.

  
  

  torch/_C.cpython-312-x86_64-linux-gnu.so   22,960  ->  15,720   (-7,240)
  
  

  
  

  Every other one of 26,778 files is byte-identical in size. That is
  
  precisely the file the PR rebuilds via CMake.

  
  


• 
  

  torch._C is functionally equivalent between parent and PR.
  
  Extracted both .so files and compared:

  
  
  
  
  • Defined symbols: zero-line diff.
  
  
  • Undefined symbols (imports from libtorch_python): zero-line
    
    diff.
  
  
  • RPATH: identical ($ORIGIN:$ORIGIN/lib).
  
  
  • NEEDED: 180242's .so includes a redundant
    
    libpthread.so.0 (merged into libc.so.6 since glibc 2.34); the
    
    CMake build drops it. The size shrinkage matches the cleaner
    
    NEEDED + tighter section layout from the CMake toolchain.
  
  
  
  


• 
  

  torch/version.py is present in both wheels at 339 bytes,
  
  byte-identical except for the expected git_version SHA (each
  
  commit has its own). __version__, debug, cuda, hip, rocm,
  
  xpu fields all match -- the CMake-driven generation produces the
  
  same output as the setuptools path.

  
  


• 
  

  Exactly one _C ABI per wheel. On a fresh multi-ABI
  
  manywheel-cpu-build (build_all.sh builds all CPython ABIs in one
  
  checkout), each wheel ships only its own torch/_C.cpython-XY.so.
  
  An earlier greedy _C*.so package_data glob leaked stale per-ABI
  
  stubs (a cp312 wheel carried cp310/cp311/cp312); pinning to the
  
  exact EXT_SUFFIX fixes it.

  
  

  unzip -l manywheel-py3_13-cpu.whl | grep 'torch/_C\.cpython'
  
  # -> only torch/_C.cpython-313-x86_64-linux-gnu.so
  
  

  
  

Coverage notes

• NNAPI MSVC link-flag fix and Windows venv pythonXX.lib
  
  discovery fix are not exercised by the ROCm-Linux artifacts
  
  above. They are exercised by the Windows pull-CI (where NNAPI
  
  builds run and venv-based discovery is relevant); the green Windows
  
  CI is the evidence for those.


• ROCm wheels are Linux-only by build matrix
  
  (.github/scripts/generate_binary_build_matrix.py:130-135,
  
  supported_platforms=["Linux"]).

Co-Generated with Claude Code

Pull Request resolved: #180243

Approved by: https://github.com/malfet