Source Code Link
🗳️ ObjWatch is a Python library to trace and monitor object attributes and method calls.
ObjWatch
]
Overview
ObjWatch is a robust Python library designed to streamline the debugging and monitoring of complex projects. By offering real-time tracing of object attributes and method calls, ObjWatch empowers developers to gain deeper insights into their codebases, facilitating issue identification, performance optimization, and overall code quality enhancement.
⚠️ Performance Warning
ObjWatch may impact your application's performance. It is recommended to use it solely in debugging environments.
Features
Nested Structure Tracing: Visualize and monitor nested function calls and object interactions with clear, hierarchical logging.
Enhanced Logging Support: Leverage Python's built-in
loggingmodule for structured, customizable log outputs, including support for simple and detailed formats. Additionally, to ensure logs are captured even if the logger is disabled or removed by external libraries, you can setlevel="force". Whenlevelis set to"force", ObjWatch bypasses the standard logging handlers and usesprint()to…
TensorShapeLogger
This is an example of a custom wrapper I implemented based on my usage scenario. The code is in the objwatch/wrappers.py file. This wrapper automatically records the tensor shapes of inputs and outputs in all function method calls within the specified module, as well as the states of variables. This is extremely useful for understanding the execution logic of complex distributed frameworks.
class TensorShapeLogger(FunctionWrapper):
"""
TensorShapeLogger extends FunctionWrapper to log the shapes of torch.Tensor objects.
"""
@staticmethod
def _process_tensor_item(seq: List[Any]) -> Optional[List[Any]]:
"""
Process a sequence to extract tensor shapes if all items are torch.Tensor.
Args:
seq (List[Any]): The sequence to process.
Returns:
Optional[List[Any]]: List of tensor shapes or None if not applicable.
"""
if torch is not None and all(isinstance(x, torch.Tensor) for x in seq):
return [x.shape for x in seq]
else:
return None
def wrap_call(self, func_name: str, frame: FrameType) -> str:
"""
Format the function call information, including tensor shapes if applicable.
Args:
func_name (str): Name of the function being called.
frame (FrameType): The current stack frame.
Returns:
str: Formatted call message.
"""
args, kwargs = self._extract_args_kwargs(frame)
call_msg = self._format_args_kwargs(args, kwargs)
return call_msg
def wrap_return(self, func_name: str, result: Any) -> str:
"""
Format the function return information, including tensor shapes if applicable.
Args:
func_name (str): Name of the function returning.
result (Any): The result returned by the function.
Returns:
str: Formatted return message.
"""
return_msg = self._format_return(result)
return return_msg
def wrap_upd(self, old_value: Any, current_value: Any) -> Tuple[str, str]:
"""
Format the update information of a variable, including tensor shapes if applicable.
Args:
old_value (Any): The old value of the variable.
current_value (Any): The new value of the variable.
Returns:
Tuple[str, str]: Formatted old and new values.
"""
old_msg = self._format_value(old_value)
current_msg = self._format_value(current_value)
return old_msg, current_msg
def _format_value(self, value: Any, is_return: bool = False) -> str:
"""
Format a value into a string, logging tensor shapes if applicable.
Args:
value (Any): The value to format.
is_return (bool): Flag indicating if the value is a return value.
Returns:
str: Formatted value string.
"""
if torch is not None and isinstance(value, torch.Tensor):
formatted = f"{value.shape}"
elif isinstance(value, log_element_types):
formatted = f"{value}"
elif isinstance(value, log_sequence_types):
formatted_sequence = EventHandls.format_sequence(value, func=TensorShapeLogger._process_tensor_item)
if formatted_sequence:
formatted = f"{formatted_sequence}"
else:
formatted = f"(type){value.__class__.__name__}"
else:
formatted = f"(type){value.__class__.__name__}"
if is_return:
if isinstance(value, torch.Tensor):
return f"{value.shape}"
elif isinstance(value, log_sequence_types) and formatted:
return f"[{formatted}]"
return f"{formatted}"
return formatted
In deep learning projects, the shape and dimensions of tensors are crucial. A small dimension error can prevent the entire model from training or predicting correctly. Manually checking each tensor's shape is tedious and error-prone. The TensorShapeLogger automates the recording of tensor shapes, helping developers to:
- Quickly identify dimension mismatch issues: Automatically records shape information to promptly detect and fix dimension errors.
- Optimize model architecture: By tracking the changes in tensor shapes, optimize the network structure to improve model performance.
- Increase debugging efficiency: Reduce the time spent manually checking tensor shapes, allowing focus on core model development.
Example of Using a Custom Wrapper
It is recommended to refer to the tests/test_torch_train.py file. This file contains a complete example of a PyTorch training process, demonstrating how to integrate ObjWatch for monitoring and logging.
Notes
⚠️ Performance Warning
ObjWatch can impact the performance of your program when used in a debugging environment. Therefore, it is recommended to use it only during the debugging and development phases.
This is just an initial write-up; I plan to add more over time. If you find it useful, feel free to give it a star.
The library is still actively being updated. If you have any questions or suggestions, please leave a comment or open an issue in the repository.
Community-Analysen & Experten-Meinungen 0
Verwandte Story-Cluster & Quellen (Vektor-KI)
Ähnliche Beiträge
Auch interessante Nachrichten Debugging Savior! Leveraging ObjWatch for Efficient Code Comprehension and Debugging in Complex Python Projects
Thematisch verwandte Begriffe: Debugging, Savior, Leveraging, ObjWatch · 6 Treffer
PARALLAX Payload Extractor
Proofpoint Brings OpenAI GPT Cyber Models into Security Operations to Help Defenders Investigate Threats Faster
ICEDID Configuration Extractor
QBOT Configuration Extractor
BPFDoor Configuration Extractor
BPFDoor Scanner
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
Beiträge werden geladen ...
Videos werden geladen ...
SOCIAL SHARE CARD GENERATOR