…API (#57977)

## Summary

This PR updates the document embedding benchmark to use the canonical
Ray Data implementation pattern, following best practices for the
framework.

## Key Changes

### Use `download()` expression instead of separate materialization
**Before:**
```python
file_paths = (
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .take_all()
)
file_paths = [row["uploaded_pdf_path"] for row in file_paths]
ds = ray.data.read_binary_files(file_paths, include_paths=True)
```

**After:**
```python
(
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .with_column("bytes", download("uploaded_pdf_path"))
```

This change:
- Eliminates the intermediate materialization with `take_all()`, which
loads all data into memory
- Uses the `download()` expression to lazily fetch file contents as part
of the pipeline
- Removes the need for a separate `read_binary_files()` call

### Method chaining for cleaner code
All operations are now chained in a single pipeline, making the data
flow more clear and idiomatic.

### Consistent column naming
Updated references from `path` to `uploaded_pdf_path` throughout the
code for consistency with the source data schema.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>

…API (ray-project#57977)

## Summary

This PR updates the document embedding benchmark to use the canonical
Ray Data implementation pattern, following best practices for the
framework.

## Key Changes

### Use `download()` expression instead of separate materialization
**Before:**
```python
file_paths = (
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .take_all()
)
file_paths = [row["uploaded_pdf_path"] for row in file_paths]
ds = ray.data.read_binary_files(file_paths, include_paths=True)
```

**After:**
```python
(
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .with_column("bytes", download("uploaded_pdf_path"))
```

This change:
- Eliminates the intermediate materialization with `take_all()`, which
loads all data into memory
- Uses the `download()` expression to lazily fetch file contents as part
of the pipeline
- Removes the need for a separate `read_binary_files()` call

### Method chaining for cleaner code
All operations are now chained in a single pipeline, making the data
flow more clear and idiomatic.

### Consistent column naming
Updated references from `path` to `uploaded_pdf_path` throughout the
code for consistency with the source data schema.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>

…API (ray-project#57977)

## Summary

This PR updates the document embedding benchmark to use the canonical
Ray Data implementation pattern, following best practices for the
framework.

## Key Changes

### Use `download()` expression instead of separate materialization
**Before:**
```python
file_paths = (
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .take_all()
)
file_paths = [row["uploaded_pdf_path"] for row in file_paths]
ds = ray.data.read_binary_files(file_paths, include_paths=True)
```

**After:**
```python
(
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .with_column("bytes", download("uploaded_pdf_path"))
```

This change:
- Eliminates the intermediate materialization with `take_all()`, which
loads all data into memory
- Uses the `download()` expression to lazily fetch file contents as part
of the pipeline
- Removes the need for a separate `read_binary_files()` call

### Method chaining for cleaner code
All operations are now chained in a single pipeline, making the data
flow more clear and idiomatic.

### Consistent column naming
Updated references from `path` to `uploaded_pdf_path` throughout the
code for consistency with the source data schema.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: Aydin Abiar <aydin@anyscale.com>

…API (ray-project#57977)

## Summary

This PR updates the document embedding benchmark to use the canonical
Ray Data implementation pattern, following best practices for the
framework.

## Key Changes

### Use `download()` expression instead of separate materialization
**Before:**
```python
file_paths = (
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .take_all()
)
file_paths = [row["uploaded_pdf_path"] for row in file_paths]
ds = ray.data.read_binary_files(file_paths, include_paths=True)
```

**After:**
```python
(
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .with_column("bytes", download("uploaded_pdf_path"))
```

This change:
- Eliminates the intermediate materialization with `take_all()`, which
loads all data into memory
- Uses the `download()` expression to lazily fetch file contents as part
of the pipeline
- Removes the need for a separate `read_binary_files()` call

### Method chaining for cleaner code
All operations are now chained in a single pipeline, making the data
flow more clear and idiomatic.

### Consistent column naming
Updated references from `path` to `uploaded_pdf_path` throughout the
code for consistency with the source data schema.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: Future-Outlier <eric901201@gmail.com>

…API (ray-project#57977)

## Summary

This PR updates the document embedding benchmark to use the canonical
Ray Data implementation pattern, following best practices for the
framework.

## Key Changes

### Use `download()` expression instead of separate materialization
**Before:**
```python
file_paths = (
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .take_all()
)
file_paths = [row["uploaded_pdf_path"] for row in file_paths]
ds = ray.data.read_binary_files(file_paths, include_paths=True)
```

**After:**
```python
(
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .with_column("bytes", download("uploaded_pdf_path"))
```

This change:
- Eliminates the intermediate materialization with `take_all()`, which
loads all data into memory
- Uses the `download()` expression to lazily fetch file contents as part
of the pipeline
- Removes the need for a separate `read_binary_files()` call

### Method chaining for cleaner code
All operations are now chained in a single pipeline, making the data
flow more clear and idiomatic.

### Consistent column naming
Updated references from `path` to `uploaded_pdf_path` throughout the
code for consistency with the source data schema.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>

…API (ray-project#57977)

## Summary

This PR updates the document embedding benchmark to use the canonical
Ray Data implementation pattern, following best practices for the
framework.

## Key Changes

### Use `download()` expression instead of separate materialization
**Before:**
```python
file_paths = (
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .take_all()
)
file_paths = [row["uploaded_pdf_path"] for row in file_paths]
ds = ray.data.read_binary_files(file_paths, include_paths=True)
```

**After:**
```python
(
    ray.data.read_parquet(INPUT_PATH)
    .filter(lambda row: row["file_name"].endswith(".pdf"))
    .with_column("bytes", download("uploaded_pdf_path"))
```

This change:
- Eliminates the intermediate materialization with `take_all()`, which
loads all data into memory
- Uses the `download()` expression to lazily fetch file contents as part
of the pipeline
- Removes the need for a separate `read_binary_files()` call

### Method chaining for cleaner code
All operations are now chained in a single pipeline, making the data
flow more clear and idiomatic.

### Consistent column naming
Updated references from `path` to `uploaded_pdf_path` throughout the
code for consistency with the source data schema.

Signed-off-by: Balaji Veeramani <bveeramani@berkeley.edu>
Signed-off-by: peterxcli <peterxcli@gmail.com>