trunk/33004619eb042c0a6b85db3e2e17a6409b89219d: [DTensor] Use explicit hints for unbacked sharding (#183545)

Stacked PRs:

• #183838


• #186255


• #183839


• #183840


• #183837


• #184911


• ->#183545

[DTensor] Use explicit hints for unbacked sharding

DTensor sharding falls back to conservative behavior when a shard size is

unbacked because divisibility and sharding metadata often require Python

decisions. That is correct for arbitrary symbolic sizes, but it is too

conservative for tracing paths that explicitly override optimization hints and

know that a sharded dimension or strict-view split factor is stable.

This change only trusts explicit optimization hint overrides for those Python

decisions. When a hint selects a no-padding or shardable branch, the traced graph

still emits the symbolic torch._check for the correctness predicate, so runtime

values that satisfy the predicate continue to work and values that violate it

fail closed. The hint chooses the trace-time policy branch; it is not baked in as

a concrete tensor extent.

For _StridedShard, view propagation now preserves a symbolic split_factor when

the split factor came from symbolic shape propagation. The C++ placement still

needs an int64_t unroll count for list-based lowering, so the Python wrapper

stores the symbolic split factor separately and uses the explicit hint only as a

guarded unroll count. The equality assertion is deferred instead of directly

calling torch._check(symbol == hint), because the direct check immediately

refines the ShapeEnv replacement and destroys the symbolic split_factor

metadata we need to preserve.

Assisted by Codex.

Fixes #183678

Test Plan:

python -m pytest -q test/distributed/tensor/test_placement_types.py test/distributed/tensor/test_view_ops.py::TestViewOps::test_view_groups_unbacked_sharding_propagation

lintrunner --config=.lintrunner.toml torch/distributed/tensor/placement_types.py torch/distributed/tensor/_ops/_view_ops.py torch/distributed/tensor/_ops/utils.py test/distributed/tensor/test_placement_types.py test/distributed/tensor/test_view_ops.py

Pull Request resolved: #183545

Approved by: https://github.com/pianpwk