fix progress bar always 1 short | updated generator in examples to 4o#67
Merged
fix progress bar always 1 short | updated generator in examples to 4o#67
Conversation
Contributor
There was a problem hiding this comment.
👍 Looks good to me! Reviewed everything up to 43542cc in 1 minute and 14 seconds
More details
- Looked at
47lines of code in3files - Skipped
0files when reviewing. - Skipped posting
3drafted comments based on config settings.
1. continuous_eval/generators/simple.py:276
- Draft comment:
The update to the progress bar at the end of the generation process seems correct, but the updates within the loops (lines 228 and 254) might not accurately reflect the progress. Consider updating the progress bar directly with the number of new questions generated since the last update to ensure it reflects the correct progress throughout the generation process. - Reason this comment was not posted:
Confidence of 0% on close inspection, compared to threshold of 50%.
2. docs/src/content/docs/examples/Advanced/dataset_generation.md:25
- Draft comment:
The update to the generator model from "gpt-4-0125-preview" to "gpt-4o" in the documentation and example scripts appears consistent and intentional. Ensure that "gpt-4o" is a valid and supported model identifier in the LLMFactory. - Reason this comment was not posted:
Confidence changes required:33%
The PR also includes changes to the generator model used in the examples and the simple dataset generation script. The model has been updated from "gpt-4-0125-preview" to "gpt-4o". This change is consistent across the documentation and the example script, which suggests that it was an intentional update to use a newer or different model version. This change doesn't seem to introduce any issues as long as "gpt-4o" is a valid model identifier that is supported by the LLMFactory.
3. docs/src/content/docs/examples/Advanced/dataset_generation.md:27
- Draft comment:
There is a typo in the variable namemulti_hop_precentage; it should bemulti_hop_percentage. This typo appears in both the documentation and the example scripts. Correcting this will improve code clarity and prevent potential issues in future modifications. - Reason this comment was not posted:
Confidence of 0% on close inspection, compared to threshold of 50%.
Workflow ID: wflow_uLO4ZlgPKNwzTCwE
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
Contributor
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on a05a005 in 1 minute and 15 seconds
More details
- Looked at
12lines of code in1files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. pyproject.toml:3
- Draft comment:
The PR description mentions significant changes to the progress bar logic and updates to the generator model in example scripts, but the diff only shows a version bump inpyproject.toml. Please ensure that all intended changes are included in the PR. - Reason this comment was not posted:
Confidence of 0% on close inspection, compared to threshold of 50%.
Workflow ID: wflow_Xp64FKb9Xwd56Zuq
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
This PR updates the progress bar logic, changes the generator model, adjusts question generation parameters, and bumps the software version.
Key points:
continuous_eval/generators/simple.pyto accurately reflect question count.gpt-4-0125-previewtogpt-4oin example scripts.pyproject.tomlfrom0.3.9to0.3.10.Generated with ❤️ by ellipsis.dev