⚡️ Speed up function `_parse_project_urls` by 64% by codeflash-ai[bot] · Pull Request #2 · KRRT7/packaging

codeflash-ai · 2025-11-25T07:36:29Z

📄 64% (0.64x) speedup for `_parse_project_urls` in `src/packaging/metadata.py`

⏱️ Runtime : 2.25 microseconds → 1.38 microseconds (best of 250 runs)

📝 Explanation and details

The optimization replaces expensive string manipulation operations with more direct string operations.

Key changes:

Replaced pair.split(",", 1) followed by list comprehension and parts.extend() with a single pair.find(",") call
Eliminated intermediate list creation and the need to pad the list to ensure 2 items
Reduced from 4 operations (split, list comprehension with strip, extend, unpacking) to 2-3 operations (find, slice, strip)

Why it's faster:

str.find() is more efficient than str.split() when you only need the position of the first delimiter
Avoids creating an intermediate list and the associated memory allocation/deallocation overhead
Eliminates the max(0, 2 - len(parts)) calculation and list extension operation
Directly handles the two cases (comma found vs. not found) with conditional logic instead of post-processing a list

Performance impact:
The function is called from parse_email() when processing "project_urls" metadata fields. Since package metadata parsing can happen frequently during dependency resolution and package installation, this 63% speedup provides meaningful performance benefits. The optimization is particularly effective for the large-scale test cases with 1000+ URL pairs, where the reduced per-iteration overhead compounds significantly.

Test case effectiveness:
The optimization performs well across all test scenarios - basic cases with few URLs, edge cases with malformed data, and especially large-scale cases with hundreds of URL pairs where the reduced allocation overhead provides the most benefit.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 47 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	✅ 2 Passed
📊 Tests Coverage	100.0%

🌀 Generated Regression Tests and Runtime

from __future__ import annotations

import sys

# imports
import pytest
from src.packaging.metadata import _parse_project_urls

# unit tests

# ------------------------
# 1. Basic Test Cases
# ------------------------

def test_empty_list_returns_empty_dict():
    # Test that an empty list returns an empty dictionary
    codeflash_output = _parse_project_urls([])

def test_single_valid_pair():
    # Test a single valid label,url pair
    data = ["Homepage, https://example.com"]
    codeflash_output = _parse_project_urls(data)

def test_multiple_valid_pairs():
    # Test multiple valid pairs
    data = [
        "Homepage, https://example.com",
        "Documentation, https://docs.example.com",
        "Source, https://github.com/example/repo"
    ]
    expected = {
        "Homepage": "https://example.com",
        "Documentation": "https://docs.example.com",
        "Source": "https://github.com/example/repo"
    }
    codeflash_output = _parse_project_urls(data)

def test_whitespace_is_stripped():
    # Test that leading/trailing whitespace is stripped from label and url
    data = ["  Homepage  ,   https://example.com   "]
    codeflash_output = _parse_project_urls(data)

def test_url_with_comma():
    # Test that only the first comma splits the label and url
    data = ["Homepage, https://example.com/page,section"]
    codeflash_output = _parse_project_urls(data)

# ------------------------
# 2. Edge Test Cases
# ------------------------

def test_no_comma_in_pair():
    # Test a string with no comma: label only, url should be empty string
    data = ["Homepage"]
    codeflash_output = _parse_project_urls(data)

def test_empty_string_pair():
    # Test an empty string in the list: both label and url should be empty
    data = [""]
    codeflash_output = _parse_project_urls(data)

def test_empty_label():
    # Test a pair with empty label but present url
    data = [", https://example.com"]
    codeflash_output = _parse_project_urls(data)

def test_empty_url():
    # Test a pair with label and empty url
    data = ["Homepage, "]
    codeflash_output = _parse_project_urls(data)

def test_label_and_url_both_empty():
    # Test a pair with both label and url empty
    data = [","]
    codeflash_output = _parse_project_urls(data)

def test_duplicate_labels_raises_keyerror():
    # Test that duplicate labels raise KeyError
    data = [
        "Homepage, https://example.com",
        "Homepage, https://another.com"
    ]
    with pytest.raises(KeyError, match="duplicate labels in project urls"):
        _parse_project_urls(data)

def test_duplicate_labels_case_sensitive():
    # Labels are case sensitive, so "homepage" and "Homepage" are different
    data = [
        "Homepage, https://example.com",
        "homepage, https://another.com"
    ]
    expected = {
        "Homepage": "https://example.com",
        "homepage": "https://another.com"
    }
    codeflash_output = _parse_project_urls(data)

def test_label_with_internal_comma():
    # Label with internal comma, only first comma splits
    data = ["Home,page, https://example.com"]
    codeflash_output = _parse_project_urls(data)

def test_url_is_empty_and_label_is_whitespace():
    # Pair with label as whitespace and url as empty
    data = ["   ,"]
    codeflash_output = _parse_project_urls(data)

def test_multiple_empty_pairs():
    # Multiple empty pairs, should raise KeyError due to duplicate empty label
    data = ["", ""]
    with pytest.raises(KeyError, match="duplicate labels in project urls"):
        _parse_project_urls(data)

def test_label_is_only_whitespace():
    # Pair with label as whitespace, url present
    data = ["   , https://example.com"]
    codeflash_output = _parse_project_urls(data)

def test_url_is_only_whitespace():
    # Pair with label present, url as whitespace
    data = ["Homepage,    "]
    codeflash_output = _parse_project_urls(data)

# ------------------------
# 3. Large Scale Test Cases
# ------------------------

def test_large_number_of_unique_pairs():
    # Test with 1000 unique pairs
    data = [f"Label{i}, https://example.com/{i}" for i in range(1000)]
    expected = {f"Label{i}": f"https://example.com/{i}" for i in range(1000)}
    codeflash_output = _parse_project_urls(data)

def test_large_number_of_pairs_with_whitespace():
    # Test with 1000 pairs with extra whitespace
    data = [f"  Label{i}  ,   https://example.com/{i}   " for i in range(1000)]
    expected = {f"Label{i}": f"https://example.com/{i}" for i in range(1000)}
    codeflash_output = _parse_project_urls(data)

def test_large_number_with_one_duplicate_label():
    # Test with 999 unique pairs and 1 duplicate, should raise KeyError
    data = [f"Label{i}, https://example.com/{i}" for i in range(999)]
    data.append("Label0, https://example.com/duplicate")
    with pytest.raises(KeyError, match="duplicate labels in project urls"):
        _parse_project_urls(data)

def test_large_number_with_empty_labels():
    # Test with 999 unique pairs and 1 pair with empty label, should work unless duplicate
    data = [f"Label{i}, https://example.com/{i}" for i in range(998)]
    data.append(", https://example.com/empty")
    data.append(", https://example.com/empty2")
    # The two empty labels should cause KeyError
    with pytest.raises(KeyError, match="duplicate labels in project urls"):
        _parse_project_urls(data)

def test_large_number_of_pairs_with_no_commas():
    # Test with 1000 pairs, none with commas
    data = [f"Label{i}" for i in range(1000)]
    expected = {f"Label{i}": "" for i in range(1000)}
    codeflash_output = _parse_project_urls(data)

def test_performance_large_scale():
    # Test that function completes quickly with large input
    import time
    data = [f"Label{i}, https://example.com/{i}" for i in range(1000)]
    start = time.time()
    codeflash_output = _parse_project_urls(data); result = codeflash_output
    duration = time.time() - start
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

from typing import Dict, List

# imports
import pytest  # used for our unit tests
from src.packaging.metadata import _parse_project_urls

# unit tests

# -------------------------------
# Basic Test Cases
# -------------------------------

def test_empty_list_returns_empty_dict():
    # No data should return an empty dictionary
    codeflash_output = _parse_project_urls([])

def test_single_pair():
    # A single well-formed label,url
    codeflash_output = _parse_project_urls(["Homepage, https://example.com"])

def test_multiple_pairs():
    # Multiple well-formed pairs
    data = [
        "Homepage, https://example.com",
        "Documentation, https://docs.example.com",
        "Source, https://github.com/example/repo"
    ]
    expected = {
        "Homepage": "https://example.com",
        "Documentation": "https://docs.example.com",
        "Source": "https://github.com/example/repo"
    }
    codeflash_output = _parse_project_urls(data)

def test_spaces_are_stripped():
    # Leading/trailing spaces in label and url should be stripped
    data = [
        " Homepage  ,   https://example.com  ",
        "Docs , https://docs.example.com "
    ]
    expected = {
        "Homepage": "https://example.com",
        "Docs": "https://docs.example.com"
    }
    codeflash_output = _parse_project_urls(data)

def test_label_with_comma_in_url():
    # Only the first comma is split, so URLs with commas are ok
    data = [
        "Homepage, https://example.com/path,with,commas"
    ]
    expected = {
        "Homepage": "https://example.com/path,with,commas"
    }
    codeflash_output = _parse_project_urls(data)

# -------------------------------
# Edge Test Cases
# -------------------------------

def test_missing_comma_results_in_empty_url():
    # If there's no comma, label is the whole string, url is ''
    data = ["Homepage"]
    expected = {"Homepage": ""}
    codeflash_output = _parse_project_urls(data)

def test_empty_label_nonempty_url():
    # If the string starts with a comma, label is '', url is the rest
    data = [",https://example.com"]
    expected = {"": "https://example.com"}
    codeflash_output = _parse_project_urls(data)

def test_empty_label_and_url():
    # If the string is just a comma, both label and url are ''
    data = [","]
    expected = {"": ""}
    codeflash_output = _parse_project_urls(data)

def test_empty_string():
    # If the string is empty, label is '', url is ''
    data = [""]
    expected = {"": ""}
    codeflash_output = _parse_project_urls(data)

def test_duplicate_labels_raises_keyerror():
    # Duplicate labels (after stripping) should raise KeyError
    data = [
        "Homepage, https://example.com",
        "Homepage, https://other.com"
    ]
    with pytest.raises(KeyError):
        _parse_project_urls(data)

def test_duplicate_labels_with_spaces_raises_keyerror():
    # Duplicate labels with different spacing should also raise KeyError
    data = [
        "Homepage, https://example.com",
        "  Homepage   , https://other.com"
    ]
    with pytest.raises(KeyError):
        _parse_project_urls(data)

def test_duplicate_empty_label_raises_keyerror():
    # Two entries with empty label should raise KeyError
    data = [
        ",https://a.com",
        ",https://b.com"
    ]
    with pytest.raises(KeyError):
        _parse_project_urls(data)

def test_label_case_sensitive():
    # Labels are case sensitive, so "HomePage" and "homepage" are different
    data = [
        "HomePage, https://a.com",
        "homepage, https://b.com"
    ]
    expected = {
        "HomePage": "https://a.com",
        "homepage": "https://b.com"
    }
    codeflash_output = _parse_project_urls(data)

def test_label_with_only_spaces():
    # Label with only spaces is stripped to '', so two such entries are duplicate
    data = [
        "   , https://a.com",
        "   , https://b.com"
    ]
    with pytest.raises(KeyError):
        _parse_project_urls(data)

def test_url_with_leading_trailing_spaces():
    # URL with spaces should be stripped
    data = [
        "Homepage,   https://example.com   "
    ]
    expected = {"Homepage": "https://example.com"}
    codeflash_output = _parse_project_urls(data)

def test_label_with_leading_trailing_spaces():
    # Label with spaces should be stripped
    data = [
        "   Homepage   ,https://example.com"
    ]
    expected = {"Homepage": "https://example.com"}
    codeflash_output = _parse_project_urls(data)

def test_label_and_url_both_empty():
    # Both label and url are empty after stripping
    data = ["  ,   "]
    expected = {"": ""}
    codeflash_output = _parse_project_urls(data)

def test_label_with_comma_in_label():
    # Only the first comma splits, so label can contain commas if not the first
    data = [
        "Label, with, comma, https://example.com"
    ]
    expected = {
        "Label": "with, comma, https://example.com"
    }
    codeflash_output = _parse_project_urls(data)

# -------------------------------
# Large Scale Test Cases
# -------------------------------

def test_large_number_of_unique_labels():
    # 1000 unique labels should be handled without error
    data = [f"Label{i}, https://example.com/{i}" for i in range(1000)]
    expected = {f"Label{i}": f"https://example.com/{i}" for i in range(1000)}
    codeflash_output = _parse_project_urls(data)

def test_large_number_of_empty_labels():
    # 2 empty labels should raise KeyError (but only test with 2 for edge)
    data = [",https://a.com", ",https://b.com"]
    with pytest.raises(KeyError):
        _parse_project_urls(data)

def test_large_number_of_duplicate_labels_raises():
    # 999 unique, 1 duplicate label at the end should raise KeyError
    data = [f"Label{i}, https://example.com/{i}" for i in range(999)]
    data.append("Label0, https://duplicate.com")
    with pytest.raises(KeyError):
        _parse_project_urls(data)

def test_large_number_of_labels_with_varied_spacing():
    # 1000 unique labels with varied spacing should be parsed correctly
    data = [f"  Label{i}   ,   https://example.com/{i}   " for i in range(1000)]
    expected = {f"Label{i}": f"https://example.com/{i}" for i in range(1000)}
    codeflash_output = _parse_project_urls(data)

def test_large_number_of_labels_with_commas_in_url():
    # 1000 unique labels, each URL contains commas
    data = [f"Label{i}, https://example.com/path,with,commas/{i}" for i in range(1000)]
    expected = {f"Label{i}": f"https://example.com/path,with,commas/{i}" for i in range(1000)}
    codeflash_output = _parse_project_urls(data)

def test_large_number_of_labels_with_commas_in_label_part():
    # Only the first comma splits, so label is always "Label{i}"
    data = [f"Label{i}, with, commas, https://example.com/{i}" for i in range(1000)]
    expected = {f"Label{i}": f"with, commas, https://example.com/{i}" for i in range(1000)}
    codeflash_output = _parse_project_urls(data)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

from src.packaging.metadata import _parse_project_urls
import pytest

def test__parse_project_urls():
    with pytest.raises(KeyError, match="'duplicate\\ labels\\ in\\ project\\ urls'"):
        _parse_project_urls(['', ''])

def test__parse_project_urls_2():
    _parse_project_urls([])

🔎 Concolic Coverage Tests and Runtime

Test File::Test Function	Original ⏱️	Optimized ⏱️	Speedup
`codeflash_concolic_quk6vk0y/tmpvw7gw64d/test_concolic_coverage.py::test__parse_project_urls`	2.00μs	1.12μs	77.8%✅
`codeflash_concolic_quk6vk0y/tmpvw7gw64d/test_concolic_coverage.py::test__parse_project_urls_2`	250ns	250ns	0.000%✅

To edit these changes git checkout codeflash/optimize-_parse_project_urls-mie9i6nb and push.

The optimization replaces expensive string manipulation operations with more direct string operations. **Key changes:** - Replaced `pair.split(",", 1)` followed by list comprehension and `parts.extend()` with a single `pair.find(",")` call - Eliminated intermediate list creation and the need to pad the list to ensure 2 items - Reduced from 4 operations (split, list comprehension with strip, extend, unpacking) to 2-3 operations (find, slice, strip) **Why it's faster:** - `str.find()` is more efficient than `str.split()` when you only need the position of the first delimiter - Avoids creating an intermediate list and the associated memory allocation/deallocation overhead - Eliminates the `max(0, 2 - len(parts))` calculation and list extension operation - Directly handles the two cases (comma found vs. not found) with conditional logic instead of post-processing a list **Performance impact:** The function is called from `parse_email()` when processing "project_urls" metadata fields. Since package metadata parsing can happen frequently during dependency resolution and package installation, this 63% speedup provides meaningful performance benefits. The optimization is particularly effective for the large-scale test cases with 1000+ URL pairs, where the reduced per-iteration overhead compounds significantly. **Test case effectiveness:** The optimization performs well across all test scenarios - basic cases with few URLs, edge cases with malformed data, and especially large-scale cases with hundreds of URL pairs where the reduced allocation overhead provides the most benefit.

henryiii · 2025-12-09T15:43:47Z

This is a great find, a poorly written expression, but came up with a terrible solution (unreadable). Giving the same three lines to ChatGPT, with a prompt: Can this be simplified? It's a bit slow, but I don't want to give up too much readability: found I think the correct fix:

label, _, url = (s.strip() for s in pair.partition(","))

I'd be curious to know how this compares.

KRRT7 · 2025-12-10T01:27:30Z

merged with changes upstream

codeflash-ai bot requested a review from KRRT7 November 25, 2025 07:36

codeflash-ai bot added ⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: High Optimization Quality according to Codeflash labels Nov 25, 2025

henryiii mentioned this pull request Dec 9, 2025

chore: use partition in _parse_project_urls pypa/packaging#1013

Merged

KRRT7 closed this Dec 10, 2025

codeflash-ai bot deleted the codeflash/optimize-_parse_project_urls-mie9i6nb branch December 10, 2025 01:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⚡️ Speed up function `_parse_project_urls` by 64%#2

⚡️ Speed up function `_parse_project_urls` by 64%#2
codeflash-ai[bot] wants to merge 1 commit intooptimization-attemptfrom
codeflash/optimize-_parse_project_urls-mie9i6nb

codeflash-ai bot commented Nov 25, 2025

Uh oh!

henryiii commented Dec 9, 2025

Uh oh!

KRRT7 commented Dec 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

codeflash-ai bot commented Nov 25, 2025

📄 64% (0.64x) speedup for _parse_project_urls in src/packaging/metadata.py

📝 Explanation and details

Uh oh!

henryiii commented Dec 9, 2025

Uh oh!

KRRT7 commented Dec 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

📄 64% (0.64x) speedup for `_parse_project_urls` in `src/packaging/metadata.py`