fix(search): Consider all non-loopback IPv4s when selecting listen address to receive results (fixes #1316).#1317
Conversation
There was a problem hiding this comment.
This was auto-updated by poetry add netifaces inside components/clp-package-utils
WalkthroughAdds a psutil-based helper to resolve the host IPv4 address, replaces inline IP discovery in the search script to use that helper with logging and None handling, and declares psutil as a runtime dependency in the package pyproject. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Caller
participant Search as search.py
participant Helper as _get_ipv4_address()
participant Psutil as psutil
participant Logger as logger
Caller->>Search: do_search_without_aggregation()
Search->>Helper: request IPv4 resolution
Helper->>Psutil: query interfaces → addresses
Psutil-->>Helper: interfaces map
alt non-loopback IPv4 found
Helper-->>Search: return non-loopback IPv4
Search->>Logger: debug "Listening on resolved host"
else only loopback found
Helper-->>Search: return loopback IPv4
Search->>Logger: warning "Only loopback found"
else none found
Helper-->>Search: return None
Search->>Logger: error "No IPv4 address found"
end
Search->>Search: bind/listen on returned host (if any)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: ASSERTIVE Plan: Pro 📒 Files selected for processing (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
🔇 Additional comments (4)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Ruff (0.13.1)components/clp-package-utils/clp_package_utils/scripts/native/search.py�[1;31mruff failed�[0m Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
components/clp-package-utils/poetry.lockis excluded by!**/*.lock
📒 Files selected for processing (2)
components/clp-package-utils/clp_package_utils/scripts/native/search.py(3 hunks)components/clp-package-utils/pyproject.toml(1 hunks)
🔇 Additional comments (4)
components/clp-package-utils/clp_package_utils/scripts/native/search.py (3)
11-11: LGTM!The import of netifaces aligns with the dependency addition and is properly placed with other third-party imports.
153-157: LGTM!The replacement of the previous IP discovery logic with the new helper function is well-implemented. The None-check provides proper error handling, and the early return prevents attempting to start a server without a valid address.
157-157: LGTM!The debug logging provides valuable visibility into the selected host address, which will be helpful for troubleshooting network connectivity issues in Docker environments.
components/clp-package-utils/pyproject.toml (1)
15-15: Verified — netifaces 0.11.0 is available and compatible with Python 3.9. netifaces 0.11.0 publishes cp39 wheels (manylinux/macOS); keep the ^0.11.0 constraint.
| def _get_ipv4_address() -> str | None: | ||
| """ | ||
| Retrieves an IPv4 address of the host for network communication. | ||
|
|
||
| :returns: The first non-local IPv4 address it finds. | ||
| If no non-local address is available, returns the first local IPv4 address. | ||
| If no IPv4 address is found, returns None. | ||
| """ | ||
| local_ip = None | ||
|
|
||
| for interface in netifaces.interfaces(): | ||
| for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []): | ||
| ip = link["addr"] | ||
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): | ||
| return ip | ||
| if local_ip is None: | ||
| local_ip = ip | ||
|
|
||
| logger.warning("Couldn't find a non-local IP address for receiving search results.") | ||
| return local_ip |
There was a problem hiding this comment.
🧹 Nitpick
Improve error handling and address potential edge cases.
The function implementation is solid overall, but there are a few areas for improvement:
- Line 50: The IPv4Network constructor could raise
AddressValueErrororNetmaskValueErrorfor invalid IP addresses returned by netifaces - Lines 47-48: No error handling for potential exceptions from
netifaces.interfaces()ornetifaces.ifaddresses() - Line 55: The warning message could be more descriptive about fallback behaviour
Apply this diff to add proper error handling:
def _get_ipv4_address() -> str | None:
"""
Retrieves an IPv4 address of the host for network communication.
:returns: The first non-local IPv4 address it finds.
If no non-local address is available, returns the first local IPv4 address.
If no IPv4 address is found, returns None.
"""
local_ip = None
- for interface in netifaces.interfaces():
- for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []):
- ip = link["addr"]
- if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"):
- return ip
- if local_ip is None:
- local_ip = ip
+ try:
+ for interface in netifaces.interfaces():
+ for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []):
+ ip = link["addr"]
+ try:
+ if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"):
+ return ip
+ if local_ip is None:
+ local_ip = ip
+ except (ipaddress.AddressValueError, ipaddress.NetmaskValueError):
+ logger.debug(f"Skipping invalid IP address: {ip}")
+ continue
+ except Exception as e:
+ logger.warning(f"Failed to enumerate network interfaces: {e}")
- logger.warning("Couldn't find a non-local IP address for receiving search results.")
+ if local_ip is not None:
+ logger.warning("Couldn't find a non-local IP address for receiving search results. Falling back to local address.")
+ else:
+ logger.warning("Couldn't find any IPv4 address for receiving search results.")
return local_ip📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def _get_ipv4_address() -> str | None: | |
| """ | |
| Retrieves an IPv4 address of the host for network communication. | |
| :returns: The first non-local IPv4 address it finds. | |
| If no non-local address is available, returns the first local IPv4 address. | |
| If no IPv4 address is found, returns None. | |
| """ | |
| local_ip = None | |
| for interface in netifaces.interfaces(): | |
| for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []): | |
| ip = link["addr"] | |
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): | |
| return ip | |
| if local_ip is None: | |
| local_ip = ip | |
| logger.warning("Couldn't find a non-local IP address for receiving search results.") | |
| return local_ip | |
| def _get_ipv4_address() -> str | None: | |
| """ | |
| Retrieves an IPv4 address of the host for network communication. | |
| :returns: The first non-local IPv4 address it finds. | |
| If no non-local address is available, returns the first local IPv4 address. | |
| If no IPv4 address is found, returns None. | |
| """ | |
| local_ip = None | |
| try: | |
| for interface in netifaces.interfaces(): | |
| for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []): | |
| ip = link["addr"] | |
| try: | |
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): | |
| return ip | |
| if local_ip is None: | |
| local_ip = ip | |
| except (ipaddress.AddressValueError, ipaddress.NetmaskValueError): | |
| logger.debug(f"Skipping invalid IP address: {ip}") | |
| continue | |
| except Exception as e: | |
| logger.warning(f"Failed to enumerate network interfaces: {e}") | |
| if local_ip is not None: | |
| logger.warning("Couldn't find a non-local IP address for receiving search results. Falling back to local address.") | |
| else: | |
| logger.warning("Couldn't find any IPv4 address for receiving search results.") | |
| return local_ip |
🤖 Prompt for AI Agents
In components/clp-package-utils/clp_package_utils/scripts/native/search.py
around lines 37 to 56, add robust error handling: wrap calls to
netifaces.interfaces() and netifaces.ifaddresses() in a try/except to catch and
log any netifaces-related exceptions and continue; when processing each link,
wrap ipaddress.ip_address(ip) and the IPv4Network check in a try/except to catch
AddressValueError/NetmaskValueError (or ValueError) and log/debug the invalid
IP, then skip that entry; preserve selecting the first non-loopback IPv4 and
first local fallback; finally update the logger.warning to a more descriptive
message that states no non-local IPv4 was found and that the function is
returning the discovered local_ip fallback (or None if none found).
sitaowang1998
left a comment
There was a problem hiding this comment.
My major concern is the use of netifaces library. The library has been abandoned in 2021. @kirkrodrigues Should we look for other libraries?
| import netifaces | ||
| import msgpack |
There was a problem hiding this comment.
Nit: Sort the imports in lexical order.
| import netifaces | |
| import msgpack | |
| import msgpack | |
| import netifaces |
| for interface in netifaces.interfaces(): | ||
| for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []): | ||
| ip = link["addr"] | ||
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): |
There was a problem hiding this comment.
There are more local IPv4 address than 127.0.0.0/8. Could you add other common ones like 10.0.0.0/8 and 192.168.0.0/16.
There was a problem hiding this comment.
Here we want to obtain an address that another host (in our case, the query worker) can read. We do want to allow the query worker and the search script host to be run under a same network like 192.168.x.x.
I believe a more accurate terminology for any 127.0.0.0/8 address is "loopback" address. Let me update the descriptions and docstrings.
| logger = logging.getLogger(__file__) | ||
|
|
||
|
|
||
| def _get_ipv4_address() -> str | None: |
There was a problem hiding this comment.
Better put the private helpers after all public functions.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
components/clp-package-utils/clp_package_utils/scripts/native/search.py(3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: package-image
- GitHub Check: lint-check (macos-15)
🔇 Additional comments (3)
components/clp-package-utils/clp_package_utils/scripts/native/search.py (3)
41-43: Intent check: keep RFC1918 addresses eligible as “non‑local”.Please confirm “non‑local” here means “non‑loopback”. Excluding RFC1918 ranges (10/8, 172.16/12, 192.168/16) would break container/VPN scenarios; the current implementation correctly treats them as valid candidates.
12-12: netifaces declared in pyproject.toml — OKFound declaration: pyproject.toml (line 15): netifaces = "^0.11.0".
37-56: Harden IP detection: catch netifaces/ipaddress errors, skip link‑local, and clarify fallbacks.Uncaught exceptions (interface enumeration, transient iface removal, malformed addr) can abort the search flow. Also, treating IPv4 link‑local (169.254/16) as “non‑local” is risky; prefer skipping those and falling back to loopback if needed. Improve warnings to state the actual fallback.
Apply:
def _get_ipv4_address() -> str | None: @@ - local_ip = None - - for interface in netifaces.interfaces(): - for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []): - ip = link["addr"] - if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): - return ip - if local_ip is None: - local_ip = ip - - logger.warning("Couldn't find a non-local IP address for receiving search results.") - return local_ip + local_ip = None + try: + for interface in netifaces.interfaces(): + try: + entries = netifaces.ifaddresses(interface).get(netifaces.AF_INET, []) + except Exception as e: + logger.debug(f"Skipping interface {interface}: {e}") + continue + for link in entries: + ip = link.get("addr") + if not ip: + continue + try: + addr = ipaddress.ip_address(ip) + except ValueError: + logger.debug(f"Skipping invalid IPv4 address from {interface}: {ip}") + continue + # Prefer non-loopback, non-link-local addresses for worker connectivity. + if addr.is_loopback: + if local_ip is None: + local_ip = ip + continue + if addr.is_link_local: + # Avoid selecting 169.254.0.0/16 unless nothing else exists. + if local_ip is None: + local_ip = ip + continue + return ip + except Exception as e: + logger.warning(f"Failed to enumerate network interfaces: {e}") + + if local_ip is not None and ipaddress.ip_address(local_ip).is_loopback: + logger.warning( + "Couldn't find a non-loopback IPv4 address for receiving search results. " + "Falling back to loopback." + ) + elif local_ip is not None: + logger.warning( + "Couldn't find a routable IPv4 address; falling back to a non-routable address." + ) + else: + logger.warning("Couldn't find any IPv4 address for receiving search results.") + return local_ip
I found that too: al45tair/netifaces#78 . The Ubuntu distos do pre install the library via package https://launchpad.net/ubuntu/+source/netifaces , so community support is sort of guaranteed. |
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
components/clp-package-utils/clp_package_utils/scripts/native/search.py(3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: package-image
- GitHub Check: lint-check (macos-15)
- GitHub Check: lint-check (ubuntu-24.04)
🔇 Additional comments (1)
components/clp-package-utils/clp_package_utils/scripts/native/search.py (1)
120-140: Fix fallback logic bug and add robust error handling in _get_ipv4_address.
- The fallback line
if ip is None: ip = ipis a no‑op; you never persist the first loopback IP. As written, the function returns the last iterated IP, contradicting the docstring.- Missing try/except around netifaces/ipaddress calls can raise and abort the search flow.
Apply:
def _get_ipv4_address() -> str | None: """ - Retrieves an IPv4 address of the host for network communication. - - :returns: The first non-loopback IPv4 address it finds. - If no non-loopback address is available, returns the first loopback IPv4 address. - If no IPv4 address is found, returns None. + Retrieve a host IPv4 address for network communication. + Preference: default-route interface's non-loopback address, otherwise the + first non-loopback across all interfaces, otherwise the first loopback. + Returns None if none found. """ - ip = None - - for interface in netifaces.interfaces(): - for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []): - ip = link["addr"] - if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): - return ip - if ip is None: - ip = ip - - logger.warning("Couldn't find a non-loopback IP address for receiving search results.") - return ip + local_ip: str | None = None + try: + # Prefer interface used by the default IPv4 route when available + gws = getattr(netifaces, "gateways", None) + if callable(gws): + default_v4 = netifaces.gateways().get("default", {}).get(netifaces.AF_INET) + if default_v4: + _, iface = default_v4 + for link in netifaces.ifaddresses(iface).get(netifaces.AF_INET, []): + ip = link.get("addr") + try: + if ip and not ipaddress.ip_address(ip).is_loopback: + return ip + except (ValueError, ipaddress.AddressValueError, ipaddress.NetmaskValueError): + logger.debug(f"Skipping invalid IP address on {iface}: {ip}") + # Scan all interfaces + for iface in netifaces.interfaces(): + for link in netifaces.ifaddresses(iface).get(netifaces.AF_INET, []): + ip = link.get("addr") + try: + if not ip: + continue + if not ipaddress.ip_address(ip).is_loopback: + return ip + if local_ip is None: + local_ip = ip + except (ValueError, ipaddress.AddressValueError, ipaddress.NetmaskValueError): + logger.debug(f"Skipping invalid IP address on {iface}: {ip}") + continue + except Exception as e: + logger.warning(f"Failed to enumerate network interfaces: {e}") + if local_ip is not None: + logger.warning("No non-loopback IPv4 address found; falling back to loopback.") + else: + logger.warning("No IPv4 address found on any interface.") + return local_ip
| import sys | ||
|
|
||
| import msgpack | ||
| import netifaces |
There was a problem hiding this comment.
🧩 Analysis chain
Reassess dependency risk and packaging for netifaces.
netifaces is effectively unmaintained and can lack wheels on newer Python/OS combos. This can break the CLI at import time and in downstream distros. At minimum: verify it’s declared in pyproject, pin to a known‑good version, and document supported Python versions. Consider a graceful fallback (e.g., try psutil/ifaddr) if import/usage fails.
Run to verify dependency wiring and references:
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Where is netifaces declared?
rg -n --hidden -C2 '\bnetifaces\b'
# Show Python versions targeted in CI (if any)
rg -n --hidden -C2 'python-version|matrix.*python' .github
# Ensure no platform-specific guards are missing
rg -n --hidden -C2 'Platform:' components | head -n 50Length of output: 12607
Action: mitigate netifaces packaging risk — pin, document supported Pythons, add runtime fallback, and CI install test.
Evidence: components/clp-package-utils/pyproject.toml declares netifaces = ^0.11.0 (components/clp-package-utils/pyproject.toml:15); poetry.lock pins netifaces 0.11.0 but lists prebuilt wheels only up to CPython 3.9 plus a source tarball; code imports netifaces in components/clp-package-utils/clp_package_utils/scripts/native/search.py (import at ~line 12; usage at ~lines 130–132); CI runs Python 3.10 and 3.11 (.github/workflows/clp-docs.yaml, clp-lint.yaml).
Required fixes:
- Pin to a known-good release (avoid loose ^ range) or explicitly document/endorse the tested netifaces version in pyproject and README.
- Add a runtime guard in search.py (try/except ImportError) and a graceful fallback (ifaddr/psutil or a safe no-op with a clear error message).
- Add a CI job that installs in a minimal environment for Python 3.10 and 3.11 (no build tools) to verify wheel availability or source buildability.
- If supporting those Pythons is required, either vendor/build wheels for supported platforms or replace netifaces with a maintained alternative.
🤖 Prompt for AI Agents
In components/clp-package-utils/clp_package_utils/scripts/native/search.py
around line 12, the direct import of netifaces creates a packaging/runtime risk
because current lock shows only prebuilt wheels up to CPython 3.9; update
pyproject.toml to pin netifaces to the exact tested release (remove the loose ^
range) and document supported Python/netifaces versions in README, add a
try/except ImportError around the netifaces import and implement a graceful
fallback (prefer a maintained alternative like ifaddr or psutil if installed,
otherwise a safe no-op that raises a clear, actionable error explaining the
missing capability), and add a CI job that installs the package in a minimal
environment on Python 3.10 and 3.11 (no build tools) to verify wheel
availability or source-buildability; if long-term support for 3.10/3.11 is
required, either vendor/build wheels for target platforms or replace netifaces
with a maintained cross-version alternative and update docs accordingly.
| def _get_ipv4_address() -> str | None: | ||
| """ | ||
| Retrieves an IPv4 address of the host for network communication. | ||
|
|
||
| :returns: The first non-loopback IPv4 address it finds. | ||
| If no non-loopback address is available, returns the first loopback IPv4 address. | ||
| If no IPv4 address is found, returns None. | ||
| """ | ||
| ip = None | ||
|
|
||
| for interface in netifaces.interfaces(): | ||
| for link in netifaces.ifaddresses(interface).get(netifaces.AF_INET, []): | ||
| ip = link["addr"] | ||
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): | ||
| return ip | ||
| if ip is None: | ||
| ip = ip | ||
|
|
||
| logger.warning("Couldn't find a non-loopback IP address for receiving search results.") | ||
| return ip | ||
|
|
There was a problem hiding this comment.
🧹 Nitpick
Prefer default-route interface to avoid Docker/bridge picks.
On multi‑NIC hosts (Docker, VPN), “first non‑loopback” can select a non‑routable bridge IP. Preferring the AF_INET default route (via netifaces.gateways()) avoids that class of failures and aligns with “address another host can reach”.
If you prefer a minimal change, keep current scan order but first attempt default-route selection before the full scan (as in the diff above). Please confirm this behaviour on hosts with: only Docker0; both Wi‑Fi and Ethernet; and VPN active.
🤖 Prompt for AI Agents
In components/clp-package-utils/clp_package_utils/scripts/native/search.py
around lines 120 to 140, prefer selecting the IPv4 address from the system
default route before falling back to the existing first-non-loopback scan: call
netifaces.gateways() to find the default AF_INET gateway and use the
interface/address associated with that gateway if present and non-loopback;
otherwise proceed with the current loop over netifaces.interfaces() and
netifaces.ifaddresses() exactly as before. Ensure you still return the first
loopback if no other IPv4 is found and keep the existing warning log. After
implementing, verify behavior on three host types: (1) host with only docker0,
(2) host with both Wi‑Fi and Ethernet, and (3) host with an active VPN,
confirming the default-route address is chosen when appropriate and fallback
scan behaves unchanged.
| host = _get_ipv4_address() | ||
| if host is None: | ||
| logger.error("Couldn't find an IPv4 address for receiving search results.") | ||
| return | ||
|
|
||
| host = ip_list[0] | ||
| for ip in ip_list: | ||
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): | ||
| host = ip | ||
| break | ||
| logger.debug(f"Listening on {host} for search results.") | ||
|
|
There was a problem hiding this comment.
🧹 Nitpick
Log the actual host:port after the server binds.
The current debug log prints only the host and precedes port allocation. Log after binding with the assigned port for actionable diagnostics.
- logger.debug(f"Listening on {host} for search results.")
+ # Defer logging until we know the assigned port.
@@
- port = int(server.sockets[0].getsockname()[1])
+ port = int(server.sockets[0].getsockname()[1])
+ logger.debug(f"Listening for search results on {host}:{port}.")Also applies to: 167-169
🤖 Prompt for AI Agents
In components/clp-package-utils/clp_package_utils/scripts/native/search.py
around lines 154-159 and similarly 167-169, the debug log currently prints the
host before the server binds and the OS may assign the port later; move the
debug logging to after the server bind/start so you can include the actual bound
host:port. After bind/start, retrieve the assigned port from the bound socket or
server object (e.g., socket.getsockname() or server.sockets[0].getsockname())
and log f"Listening on {host}:{port}" (or equivalent) so the log contains the
real host and port.
We are using the Pypi one instead of the Ubuntu one. I would suggest to use libraries like scapy or psutil. |
I agree with @sitaowang1998. Can we try to use a different library than |
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
⛔ Files ignored due to path filters (1)
components/clp-package-utils/poetry.lockis excluded by!**/*.lock
📒 Files selected for processing (2)
components/clp-package-utils/clp_package_utils/scripts/native/search.py(3 hunks)components/clp-package-utils/pyproject.toml(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: package-image
- GitHub Check: lint-check (macos-15)
- GitHub Check: lint-check (ubuntu-24.04)
🔇 Additional comments (3)
components/clp-package-utils/pyproject.toml (1)
15-15: Excellent choice of replacement for netifaces.The psutil library is actively maintained, has pre-built wheels for CPython 3.6+, and is cross-platform. This addresses the reviewers' concerns about the abandoned netifaces library whilst maintaining the same functionality for IP address discovery.
components/clp-package-utils/clp_package_utils/scripts/native/search.py (2)
12-12: LGTM - Proper library replacement.The switch from netifaces to psutil addresses the maintainer's concerns and aligns with reviewer recommendations.
156-160: Improve grammar and log the actual bound address.The current approach has two issues:
- Grammar error in Line 158: "a IPv4" should be "an IPv4"
- The debug log at Line 160 shows only the host before port assignment
Apply this diff to fix both issues:
host = _get_ipv4_address() if host is None: - logger.error("Couldn't find a IPv4 address for receiving search results.") + logger.error("Couldn't find an IPv4 address for receiving search results.") return - logger.debug(f"Listening on {host} for search results.") server = await asyncio.start_server( client_connected_cb=get_worker_connection_handler(raw_output), host=host, port=0, family=socket.AF_INET, ) port = int(server.sockets[0].getsockname()[1]) + logger.debug(f"Listening for search results on {host}:{port}.")
| def _get_ipv4_address() -> str | None: | ||
| """ | ||
| Retrieves an IPv4 address of the host for network communication. | ||
|
|
||
| :returns: The first non-loopback IPv4 address it finds. | ||
| If no non-loopback address is available, returns the first loopback IPv4 address. | ||
| If no IPv4 address is found, returns None. | ||
| """ | ||
| fallback_ip = None | ||
|
|
||
| for _, addresses in psutil.net_if_addrs().items(): | ||
| for addr in addresses: | ||
| if addr.family == socket.AF_INET: | ||
| ip = addr.address | ||
| if not ipaddress.ip_address(ip).is_loopback: | ||
| return ip | ||
| if fallback_ip is None: | ||
| fallback_ip = ip | ||
|
|
||
| if not fallback_ip: | ||
| logger.warning("Couldn't find a non-loopback IP address for receiving search results.") | ||
| return fallback_ip |
There was a problem hiding this comment.
🧹 Nitpick
Consider preference for default route interface.
On multi-interface hosts (Docker, VPN environments), selecting the "first non-loopback" address might pick a non-routable bridge IP. Consider preferring the interface associated with the default route to improve reachability.
You could enhance the implementation to first try the default route interface:
def _get_ipv4_address() -> str | None:
"""
Retrieves an IPv4 address of the host for network communication.
Prefers the address from the default route interface.
"""
fallback_ip = None
# Try to get the default route interface first
try:
gateways = psutil.net_if_stats() # This is a simplified approach
# Implementation would need to identify default route interface
# and return its non-loopback IPv4 address
except Exception:
pass
# Fallback to existing logic
# ... (rest of current implementation)This would align better with "address another host can reach" semantics, but requires additional complexity.
🤖 Prompt for AI Agents
In components/clp-package-utils/clp_package_utils/scripts/native/search.py
around lines 120-141, the current function picks the first non-loopback
interface which can pick a non-routable bridge address; change it to first
attempt to determine the address used for the default route by opening a UDP
socket to a well-known remote (e.g. 8.8.8.8) and using getsockname() to get the
local IP, validate it is an IPv4 non-loopback and return it if valid, then fall
back to the existing loop over psutil.net_if_addrs to pick the first
non-loopback or the first loopback as before; ensure exceptions from the
default-route probe are caught and logged at debug level so fallback logic still
runs and keep the existing warning when no IP is found.
🛠️ Refactor suggestion
Improve error handling and consider edge cases.
The implementation correctly addresses the original IP discovery issue using psutil. However, several areas need improvement:
- Line 134:
ipaddress.ip_address(ip)could raiseAddressValueErrorfor malformed addresses - Line 130:
psutil.net_if_addrs()could raise exceptions - Line 140: The warning message doesn't distinguish between finding only loopback vs no addresses
Apply this diff to add robust error handling:
def _get_ipv4_address() -> str | None:
"""
Retrieves an IPv4 address of the host for network communication.
:returns: The first non-loopback IPv4 address it finds.
If no non-loopback address is available, returns the first loopback IPv4 address.
If no IPv4 address is found, returns None.
"""
fallback_ip = None
- for _, addresses in psutil.net_if_addrs().items():
- for addr in addresses:
- if addr.family == socket.AF_INET:
- ip = addr.address
- if not ipaddress.ip_address(ip).is_loopback:
- return ip
- if fallback_ip is None:
- fallback_ip = ip
+ try:
+ for _, addresses in psutil.net_if_addrs().items():
+ for addr in addresses:
+ if addr.family == socket.AF_INET:
+ ip = addr.address
+ try:
+ if not ipaddress.ip_address(ip).is_loopback:
+ return ip
+ if fallback_ip is None:
+ fallback_ip = ip
+ except ipaddress.AddressValueError:
+ logger.debug(f"Skipping invalid IP address: {ip}")
+ continue
+ except Exception as e:
+ logger.warning(f"Failed to enumerate network interfaces: {e}")
- if not fallback_ip:
- logger.warning("Couldn't find a non-loopback IP address for receiving search results.")
+ if fallback_ip is not None:
+ logger.warning("Couldn't find a non-loopback IP address for receiving search results. Falling back to loopback address.")
+ else:
+ logger.warning("Couldn't find any IPv4 address for receiving search results.")
return fallback_ip📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def _get_ipv4_address() -> str | None: | |
| """ | |
| Retrieves an IPv4 address of the host for network communication. | |
| :returns: The first non-loopback IPv4 address it finds. | |
| If no non-loopback address is available, returns the first loopback IPv4 address. | |
| If no IPv4 address is found, returns None. | |
| """ | |
| fallback_ip = None | |
| for _, addresses in psutil.net_if_addrs().items(): | |
| for addr in addresses: | |
| if addr.family == socket.AF_INET: | |
| ip = addr.address | |
| if not ipaddress.ip_address(ip).is_loopback: | |
| return ip | |
| if fallback_ip is None: | |
| fallback_ip = ip | |
| if not fallback_ip: | |
| logger.warning("Couldn't find a non-loopback IP address for receiving search results.") | |
| return fallback_ip | |
| def _get_ipv4_address() -> str | None: | |
| """ | |
| Retrieves an IPv4 address of the host for network communication. | |
| :returns: The first non-loopback IPv4 address it finds. | |
| If no non-loopback address is available, returns the first loopback IPv4 address. | |
| If no IPv4 address is found, returns None. | |
| """ | |
| fallback_ip = None | |
| try: | |
| for _, addresses in psutil.net_if_addrs().items(): | |
| for addr in addresses: | |
| if addr.family == socket.AF_INET: | |
| ip = addr.address | |
| try: | |
| if not ipaddress.ip_address(ip).is_loopback: | |
| return ip | |
| if fallback_ip is None: | |
| fallback_ip = ip | |
| except ipaddress.AddressValueError: | |
| logger.debug(f"Skipping invalid IP address: {ip}") | |
| continue | |
| except Exception as e: | |
| logger.warning(f"Failed to enumerate network interfaces: {e}") | |
| if fallback_ip is not None: | |
| logger.warning("Couldn't find a non-loopback IP address for receiving search results. Falling back to loopback address.") | |
| else: | |
| logger.warning("Couldn't find any IPv4 address for receiving search results.") | |
| return fallback_ip |
🤖 Prompt for AI Agents
In components/clp-package-utils/clp_package_utils/scripts/native/search.py
around lines 120 to 141, wrap the psutil.net_if_addrs() call in a try/except to
catch and log any exceptions (and return None) so the function doesn't raise on
platform/permission errors; inside the loop catch ipaddress.AddressValueError
when parsing addr.address, log a debug/warning including the malformed value and
continue; track whether any IPv4 addresses were seen (even if all loopback)
versus none at all so you can emit distinct warnings—one warning when only
loopback addresses were found and a different warning when no IPv4 addresses
were discovered; ensure the function still returns the first non-loopback IP if
found, otherwise the first loopback, or None on fatal errors.
| """ | ||
| fallback_ip = None | ||
|
|
||
| for _, addresses in psutil.net_if_addrs().items(): |
There was a problem hiding this comment.
The implementation is now replaced with psutil
sitaowang1998
left a comment
There was a problem hiding this comment.
Generally LGTM. Some style improvements.
…de review Co-authored-by: sitaowang1998 <sitaowang1998@outlook.com>
Co-authored-by: sitaowang1998 <sitaowang1998@outlook.com>
There was a problem hiding this comment.
auto-updated by running poetry add psutil inside components/clp-package-utils
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
components/clp-package-utils/clp_package_utils/scripts/native/search.py(3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: package-image
- GitHub Check: lint-check (macos-15)
🔇 Additional comments (2)
components/clp-package-utils/clp_package_utils/scripts/native/search.py (2)
136-146: Log the actual host:port after bind (move the log).Defer the log until the OS assigns the port.
- logger.debug(f"Listening on {host} for search results.") @@ - port = int(server.sockets[0].getsockname()[1]) + port = int(server.sockets[0].getsockname()[1]) + logger.debug(f"Listening for search results on {host}:{port}.")
341-364: Harden _get_ipv4_address: prefer default-route IP and add robust error handling.Prevents crashes on malformed addresses, prefers a routable IP on multi‑NIC hosts, and makes fallback behaviour explicit.
def _get_ipv4_address() -> str | None: - """ - Retrieves an IPv4 address of the host for network communication. - - :returns: The first non-loopback IPv4 address it finds. - If no non-loopback address is available, returns the first loopback IPv4 address. - If no IPv4 address is found, returns None. - """ - fallback_ip = None - - for _, addresses in psutil.net_if_addrs().items(): - for addr in addresses: - if addr.family != socket.AF_INET: - continue - ip = addr.address - if not ipaddress.ip_address(ip).is_loopback: - return ip - if fallback_ip is None: - fallback_ip = ip - - if fallback_ip is not None: - logger.warning("Couldn't find a non-loopback IP address for receiving search results.") - return fallback_ip + """ + Retrieve an IPv4 address of the host for network communication. + Preference order: + 1) Address selected by the system's default IPv4 route (best reachability). + 2) First non-loopback IPv4 discovered on any interface (via psutil). + 3) First loopback IPv4 (fallback). + 4) None if nothing is found. + """ + # 1) Try to infer the default-route address without sending traffic. + try: + with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s: + s.connect(("8.8.8.8", 53)) # UDP connect does not transmit packets + ip = s.getsockname()[0] + try: + ip_obj = ipaddress.ip_address(ip) + if not ip_obj.is_loopback: + return ip + except ValueError as ve: + logger.debug(f"Skipping invalid default-route IP '{ip}': {ve}") + except Exception as e: + logger.debug(f"Default-route probe failed: {e}") + + fallback_ip: str | None = None + + # 2) Enumerate interfaces via psutil (if available). + try: + if psutil is not None: + for _, addresses in psutil.net_if_addrs().items(): + for addr in addresses: + if addr.family != socket.AF_INET: + continue + ip = addr.address + try: + if not ipaddress.ip_address(ip).is_loopback: + return ip + if fallback_ip is None: + fallback_ip = ip + except ValueError as ve: + logger.debug(f"Skipping invalid IP address '{ip}': {ve}") + continue + else: + logger.debug("psutil is unavailable; skipping interface enumeration.") + except Exception as e: + logger.warning(f"Failed to enumerate network interfaces: {e}") + + if fallback_ip is not None: + logger.warning( + "Couldn't find a non-loopback IPv4 address for receiving search results. Falling back to loopback." + ) + else: + logger.warning("Couldn't find any IPv4 address for receiving search results.") + return fallback_ip
| import sys | ||
|
|
||
| import msgpack | ||
| import psutil |
There was a problem hiding this comment.
🧹 Nitpick
Make psutil import optional with a graceful fallback.
Avoid hard-failing at import time on platforms where psutil isn’t present; degrade to a stdlib-only path in the helper.
-import psutil
+try:
+ import psutil # type: ignore[import-not-found]
+except Exception:
+ psutil = None # fallback: helper will handle absence gracefullyPlease also verify psutil is declared as a runtime dependency for this package (pyproject) and covered by CI on all supported Python versions.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import psutil | |
| try: | |
| import psutil # type: ignore[import-not-found] | |
| except Exception: | |
| psutil = None # fallback: helper will handle absence gracefully |
| host = _get_ipv4_address() | ||
| if host is None: | ||
| logger.error("Couldn't find an IPv4 address for receiving search results.") | ||
| return | ||
|
|
||
| host = ip_list[0] | ||
| for ip in ip_list: | ||
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): | ||
| host = ip | ||
| break | ||
| logger.debug(f"Listening on {host} for search results.") |
There was a problem hiding this comment.
Exit non‑zero when no host IP is available.
Currently we log an error and return, but main() still exits 0. Bubble a failure up.
Apply in this function:
- host = _get_ipv4_address()
- if host is None:
- logger.error("Couldn't find an IPv4 address for receiving search results.")
- return
- logger.debug(f"Listening on {host} for search results.")
+ host = _get_ipv4_address()
+ if host is None:
+ raise RuntimeError("Couldn't find an IPv4 address for receiving search results.")And update main() to return a non‑zero code:
try:
asyncio.run(
do_search(
database_config,
clp_config.results_cache,
@@
)
)
- except asyncio.CancelledError:
+ except asyncio.CancelledError:
logger.error("Search cancelled.")
return -1
+ except RuntimeError as e:
+ logger.error(str(e))
+ return -1📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| host = _get_ipv4_address() | |
| if host is None: | |
| logger.error("Couldn't find an IPv4 address for receiving search results.") | |
| return | |
| host = ip_list[0] | |
| for ip in ip_list: | |
| if ipaddress.ip_address(ip) not in ipaddress.IPv4Network("127.0.0.0/8"): | |
| host = ip | |
| break | |
| logger.debug(f"Listening on {host} for search results.") | |
| host = _get_ipv4_address() | |
| if host is None: | |
| raise RuntimeError("Couldn't find an IPv4 address for receiving search results.") | |
| try: | |
| asyncio.run( | |
| do_search( | |
| database_config, | |
| clp_config.results_cache, | |
| @@ | |
| ) | |
| ) | |
| except asyncio.CancelledError: | |
| logger.error("Search cancelled.") | |
| return -1 | |
| except RuntimeError as e: | |
| logger.error(str(e)) | |
| return -1 |
🤖 Prompt for AI Agents
In components/clp-package-utils/clp_package_utils/scripts/native/search.py
around lines 132 to 136, the code logs an error and returns when no IPv4 host is
found but main() still exits with status 0; change the early return to propagate
a failure (either raise SystemExit(1) or return a non-zero int) and update
main() so it returns an int status and the __main__ entrypoint calls
sys.exit(main()) (or re-raise/propagate the SystemExit) so the process exits
with a non-zero code when no host IP is available.
Co-authored-by: sitaowang1998 <sitaowang1998@outlook.com>
…dress to receive results (fixes y-scope#1316). (y-scope#1317) Co-authored-by: sitaowang1998 <sitaowang1998@outlook.com>
Description
netifacescross-platform library for getting IP addresses on all interfaces._get_ipv4_addressto properly retrieve all IPv4 addresses for network communication.Checklist
breaking change.
Validation performed
Repeated the reproduction steps in #1316 and now able to observed search results with the search.sh script in special network / docker environments.
Summary by CodeRabbit
Bug Fixes
Chores