Add line_search_kwargs parameter to BFGS minimizer

[shin0biX]

This PR adds a line_search_kwargs parameter to the BFGS minimizer in TensorFlow Probability, allowing users to customize the underlying Hager-Zhang line search algorithm parameters.

Changes Made

Modified Files:

1. tensorflow_probability/python/optimizer/bfgs.py
2. tensorflow_probability/python/optimizer/bfgs_utils.py

Detailed Changes:

In bfgs.py:

• Added line_search_kwargs=None parameter to the minimize function signature
• Updated function docstring to document the new parameter
• Modified initial state creation to handle None values:
  python
  kwargs['line_search_kwargs'] = line_search_kwargs if line_search_kwargs is not None else {}

In bfgs_utils.py:

• Enhanced line_search_step function to:
  • Extract line_search_kwargs from the optimizer state
  • Filter out parameters that conflict with internally set values (to prevent "multiple values for keyword argument" errors)
  • Pass filtered parameters to the Hager-Zhang line search function:
    python
    **(filtered_line_search_kwargs or {})

Features

• Backward Compatible: Existing code continues to work unchanged (line_search_kwargs=None by default)
• Full Parameter Support: Supports all Hager-Zhang line search parameters:
  • initial_step_size, value_at_initial_step, value_at_zero
  • threshold_use_approximate_wolfe_condition, shrinkage_param, expansion_param
  • sufficient_decrease_param, curvature_param, max_iterations
• Safe Parameter Handling: Automatically filters out conflicting parameters that are set internally
• Flexible Usage: Accepts None, empty dict, or any valid parameter combination

Usage Example
python
import tensorflow_probability as tfp
import tensorflow as tf

def my_loss_and_grad_fn(x):
# Your loss and gradient computation
return loss, grad

results = tfp.optimizer.bfgs_minimize(
value_and_gradients_function=my_loss_and_grad_fn,
initial_position=tf.constant([0.0, 0.0]),
line_search_kwargs={
'initial_step_size': 0.1,
'threshold_use_approximate_wolfe_condition': 1e-4,
'max_iterations': 30
}
)

Testing
Comprehensive testing was performed to verify:

• ✅ Backward compatibility (existing code unaffected)
• ✅ Basic functionality with various parameter combinations
• ✅ Edge cases (None, empty dict, conflicting parameters)
• ✅ Complex optimization problems (Rosenbrock function)
• ✅ Parameter filtering to prevent internal conflicts
• ✅ Docstring and namedtuple updates

All tests pass, confirming the implementation is correct and ready for use.

Issue Addressed
This implementation addresses #1043 by adding the requested line_search_kwargs parameter to the BFGS minimizer.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[shin0biX]

The CLA and GitHub Actions Scan are passing. The remaining
Actions Workflow Security Scan is currently awaiting maintainer approval.