ciflow/torchtitan/185416: Make new_group delegate to split_group behind a migration flag

Summary:

To unblock migrating callers from new_group to split_group, route

new_group through split_group under an opt-in flag while keeping

the legacy path as the default and warning users about the upcoming

change. This lets call sites switch over incrementally without

forcing a hard cutover.

Add torch.distributed.config.new_group_use_split_group (env var

TORCH_DIST_NEW_GROUP_USE_SPLIT_GROUP, default False). When set,

new_group routes through a thin adapter that calls split_group on

the default process group, preserving new_group's return contract

(GroupMember.NON_GROUP_MEMBER for non-members). When unset, the

legacy _new_group_with_tag path runs as before and a one-time

FutureWarning advises callers to opt in.

split_group has a narrower surface than new_group, so the adapter

raises NotImplementedError for use_local_synchronization=True and

sort_ranks=False, and ValueError if device_id conflicts with the

default group's bound device. Backend-level incompatibilities (Gloo,

MPI, mismatched backends) propagate from split_group's existing

checks. Raising (rather than silently falling back) was chosen so

callers learn which call sites still need attention as the migration

progresses.

Test Plan:

Added three tests in ProcessGroupNCCLGroupTest:

• test_new_group_delegates_to_split_group_when_flag_set verifies that
  
  comm_split_count increments and the resulting subgroup is
  
  functional, and that no migration warning fires under the flag.


• test_new_group_warns_once_when_flag_unset resets the per-process
  
  latch, calls new_group twice, and asserts exactly one
  
  FutureWarning is emitted.


• test_new_group_via_split_group_raises_on_unsupported_args asserts
  
  NotImplementedError for use_local_synchronization=True and
  
  sort_ranks=False when delegation is enabled.

Run with:

python test/distributed/test_c10d_nccl.py ProcessGroupNCCLGroupTest \

    -k 'new_group_delegates_to_split_group_when_flag_set or \

        new_group_warns_once_when_flag_unset or \

        new_group_via_split_group_raises_on_unsupported_args'

(Requires NCCL 2.18+ and 2+ GPUs; not executed locally.)

Authored by Claude.