feat(youtube): added support for new endpoints#7
feat(youtube): added support for new endpoints#7rafalzawadzki merged 3 commits intosupadata-ai:mainfrom
Conversation
WalkthroughThis pull request adds support for new YouTube endpoints in the Supadata SDK. The changes update the README with revised import orders and example code for fetching metadata for a video, channel, and playlist. New examples in the Changes
Sequence Diagram(s)sequenceDiagram
participant User as User
participant Srv as YouTubeService
participant API as YouTube API
User->>Srv: Call video(params)
Srv->>API: GET /youtube/video
API-->>Srv: Return video data
Srv-->>User: Return YoutubeVideo object
User->>Srv: Call channel(params)
Srv->>API: GET /youtube/channel
API-->>Srv: Return channel data
Srv->>Srv: Attach videos method with limit validation
Srv-->>User: Return YoutubeChannel object
User->>Srv: Call playlist(params)
Srv->>API: GET /youtube/playlist
API-->>Srv: Return playlist data
Srv->>Srv: Attach videos method with limit validation
Srv-->>User: Return YoutubePlaylist object
Assessment against linked issues
Poem
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
💵 To receive payouts, sign up on Algora, link your Github account and connect with Stripe. |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (6)
src/types.ts (1)
97-106: YoutubeChannel interface missing semicolonThe interface is well-structured with all necessary properties for a YouTube channel, but there's a missing semicolon after the handle property declaration which is inconsistent with the coding style used throughout the rest of the file.
- handle: string + handle: string;src/__tests__/supadata.test.ts (1)
2-9: Avoid shadowing the global Map typeThe static analysis tool has detected that you're shadowing the global
Maptype, which could lead to confusion. Consider renaming this import to something more specific likeSiteMapor using a qualified import.import { Supadata } from '../index.js'; import type { - Map, + Map as SiteMap, Scrape, Transcript, TranslatedTranscript, YoutubeVideo, } from '../types.js';Don't forget to update the type annotation on line 438 as well.
🧰 Tools
🪛 Biome (1.9.4)
[error] 4-4: Do not shadow the global "Map" property.
Consider renaming this variable. It's easy to confuse the origin of variables when they're named after a known global.
(lint/suspicious/noShadowRestrictedNames)
src/services/youtube.ts (4)
90-124: Clarify return type documentation for channel.videos methodThe method implementation is solid, but the JSDoc documentation should be more explicit about what the string array actually contains.
* @param params.limit - The maximum number of videos to fetch. * Default: 30. Max: 5000. -* @returns A promise that resolves to an array of strings containing the video information. +* @returns A promise that resolves to an array of video IDs (strings).
109-123: Consider extracting the limit validation logic to reduce duplicationThe same validation logic is duplicated in both
channel.videosandplaylist.videosmethods. Consider extracting this into a utility function to improve maintainability.+ // Private utility to validate limit parameter + private validateLimit(limit?: number): void { + if ( + limit === 0 || + (limit && (limit < 1 || limit > 5000)) + ) { + throw new SupadataError({ + error: 'invalid-request', + message: 'Invalid limit.', + details: 'The limit must be between 1 and 5000.', + }); + } + } channel = Object.assign( async (params: ChannelParams): Promise<YoutubeChannel> => { return this.fetch<YoutubeChannel>('/youtube/channel', params); }, { videos: async (params: ChannelVideosParams): Promise<string[]> => { // Validate the limit locally to avoid unnecessary API calls. - if ( - params.limit === 0 || - (params.limit && (params.limit < 1 || params.limit > 5000)) - ) { - throw new SupadataError({ - error: 'invalid-request', - message: 'Invalid limit.', - details: 'The limit must be between 1 and 5000.', - }); - } + this.validateLimit(params.limit); return this.fetch<string[]>('/youtube/channel/videos', params); }, } );
90-103: Add @throws documentation for consistency with playlist methodThe
channelmethod documentation is missing the@throwstag even though it can throw errors similarly to theplaylistmethod. Add this for consistency.* @param params.limit - The maximum number of videos to fetch. * Default: 30. Max: 5000. * @returns A promise that resolves to an array of strings containing the video information. +* +* @throws {SupadataError} If the limit is invalid (less than 1 or greater than 5000). */
147-159: Apply the same limit validation refactoring hereThis code duplicates the validation logic from the channel.videos method. Apply the same refactoring to use the proposed utility method.
videos: async (params: PlaylistVideosParams): Promise<string[]> => { // Validate the limit locally to avoid unnecessary API calls. - if ( - params.limit === 0 || - (params.limit && (params.limit < 1 || params.limit > 5000)) - ) { - throw new SupadataError({ - error: 'invalid-request', - message: 'Invalid limit.', - details: 'The limit must be between 1 and 5000.', - }); - } + this.validateLimit(params.limit); return this.fetch<string[]>('/youtube/playlist/videos', params); },
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (5)
README.md(2 hunks)example/index.ts(1 hunks)src/__tests__/supadata.test.ts(2 hunks)src/services/youtube.ts(3 hunks)src/types.ts(1 hunks)
🧰 Additional context used
🧬 Code Definitions (3)
example/index.ts (1)
src/services/web.ts (1) (1)
crawl(33-35)
src/__tests__/supadata.test.ts (1)
src/types.ts (1) (1)
YoutubeVideo(80-95)
src/services/youtube.ts (1)
src/types.ts (4) (4)
YoutubeVideo(80-95)YoutubeChannel(97-106)SupadataError(53-78)YoutubePlaylist(108-116)
🪛 Biome (1.9.4)
src/__tests__/supadata.test.ts
[error] 4-4: Do not shadow the global "Map" property.
Consider renaming this variable. It's easy to confuse the origin of variables when they're named after a known global.
(lint/suspicious/noShadowRestrictedNames)
🔇 Additional comments (18)
example/index.ts (6)
20-26: Well-structured example for the new YouTube video info retrievalThe example clearly demonstrates the usage of the
supadata.youtube.video()method with proper parameter passing. This matches the interface defined in the YoutubeVideo type and is consistent with the other examples in the file.
27-33: Good example for YouTube channel info retrievalThe channel info example shows how to use a URL format for the ID parameter, illustrating the flexibility of the API. This matches the method signature defined in the codebase and aligns with the documentation in the README.
34-40: Correctly implemented channel videos exampleThis example demonstrates the nested property approach for accessing channel videos via
supadata.youtube.channel.videos(), which is an intuitive API design choice. The reuse of the Fireship channel example maintains consistency.
41-47: Clear playlist info exampleThe example shows how to use a playlist ID directly, demonstrating the API's flexibility in accepting different ID formats. The comment provides helpful context about the example playlist being used.
48-54: Well-implemented playlist videos exampleThis example correctly demonstrates how to retrieve playlist videos using the nested property approach, consistent with the channel videos implementation. The example includes a helpful comment for the playlist URL.
55-61: Proper renumbering of existing examplesThe example numbering has been updated correctly to maintain sequential order after adding the new YouTube functionality examples. This ensures the examples flow logically from one to the next.
src/types.ts (2)
80-95: Well-structured YoutubeVideo interfaceThe interface provides a comprehensive representation of a YouTube video with all essential properties properly typed. The nested channel object with id and name is particularly useful for maintaining relationships between entities.
108-116: Complete YoutubePlaylist interfaceThe interface correctly defines all essential properties for a YouTube playlist with proper typing. This interface will enable strong type checking when working with playlist data returned from the API.
README.md (2)
19-29: Improved import organizationThe imports have been reordered in alphabetical order, which improves readability. The addition of the new YouTube-related types (YoutubeChannel, YoutubePlaylist, YoutubeVideo) provides clear documentation of what's available to users.
47-71: Comprehensive documentation for new YouTube endpointsThe examples for the new YouTube endpoints are clear, concise, and follow a consistent pattern. Each example includes:
- A descriptive comment
- The appropriate return type annotation
- A realistic example with options for different ID formats (URL or direct ID)
This documentation will be very helpful for users of the library.
src/__tests__/supadata.test.ts (4)
80-114: Thorough test case for video info retrievalThis test case thoroughly validates the YouTube video info functionality:
- It uses a realistic mock response that matches the YoutubeVideo interface
- It verifies both the returned data and the correct API endpoint construction
- It checks that the proper headers are sent with the request
This comprehensive testing approach will help ensure the reliability of the feature.
116-131: Good error handling test for video retrievalThe test properly verifies that error responses from the API are correctly propagated to the caller. Testing error scenarios is essential for ensuring a robust implementation.
133-266: Comprehensive channel-related testsThese tests thoroughly validate the channel info and channel videos functionality:
- Testing successful retrieval of channel info
- Testing error handling when a channel doesn't exist
- Testing channel videos retrieval with and without limits
- Testing error handling for invalid limit values
The validation of boundary conditions for the limit parameter (lines 258-270) is particularly thorough and helps ensure the API is robust against invalid inputs.
272-404: Complete test coverage for playlist functionalityThe playlist-related tests mirror the channel tests in structure, ensuring consistency across the API. They validate:
- Successful playlist info retrieval
- Error handling for non-existent playlists
- Playlist videos retrieval with and without limits
- Error handling for invalid limit values
This parallel structure in testing makes the code more maintainable and ensures consistent behavior across similar API endpoints.
src/services/youtube.ts (4)
31-45: Well-structured interface hierarchy for consistent parameter typingThe new interfaces follow a clean inheritance pattern starting from
VideoParams, which provides a good foundation for type safety across the service. This ensures consistent parameter handling for all YouTube-related API calls.
79-88: Clean implementation of the video methodThe method is well-documented with clear JSDoc comments and properly typed parameters and return values. The implementation follows the established pattern in the service.
2-9: Imports are properly organizedAll necessary types are imported correctly to support the new functionality.
126-162: Good documentation for the playlist methodThe JSDoc comments clearly describe the functionality, parameters, return values, and possible errors. The implementation follows the same pattern as the channel method, which maintains consistency.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
src/services/youtube.ts (4)
35-39: Consider refactoring the inheritance hierarchy for parameter interfacesWhile the interfaces work as implemented, there's a potential semantic issue with
ChannelParamsandPlaylistParamsextendingVideoParamssince they're not actually video parameters, they just happen to share the same structure.Consider one of these alternatives:
-export interface VideoParams { +export interface IdParams { id: string; } -export interface ChannelParams extends VideoParams {} +export interface VideoParams extends IdParams {} +export interface ChannelParams extends IdParams {} -export interface ChannelVideosParams extends ChannelParams { +export interface ChannelVideosParams extends ChannelParams { limit?: number; } -export interface PlaylistParams extends VideoParams {} +export interface PlaylistParams extends IdParams {}This would better represent that these are different types of entities that happen to share an ID structure.
Also applies to: 41-45
148-159: The validateLimit method correctly handles input validationThis helper method properly validates the limit parameter and throws an appropriate error with a descriptive message when validation fails.
Consider making this method more generic to improve reusability:
-private validateLimit(params: PlaylistVideosParams) { +private validateLimit(params: { limit?: number }) { if ( params.limit != undefined && (params.limit < 1 || params.limit > 5000) ) { throw new SupadataError({ error: 'invalid-request', message: 'Invalid limit.', details: 'The limit must be between 1 and 5000.', }); } }This would allow the method to work with any parameter object containing an optional limit property.
112-114: Consider adding validation comment about 'null' checks for limitThe current validation handles undefined values but doesn't explicitly handle null. While TypeScript types should prevent this, it might be worth adding a comment or updating the validation.
- // Validate the limit locally to avoid unnecessary API calls. + // Validate the limit locally to avoid unnecessary API calls. + // This handles both undefined and potentially null values. this.validateLimit(params);And in the validation method:
if ( - params.limit != undefined && + params.limit != null && (params.limit < 1 || params.limit > 5000) ) {Also applies to: 142-143
79-146: Add usage examples to JSDoc commentsWhile the documentation is thorough about parameters, adding usage examples would make it even more helpful for developers.
Add example usage in JSDoc comments:
/** * Fetches a YouTube video based on the provided parameters. * * @example * // Get video details * const video = await youtube.video({ id: 'dQw4w9WgXcQ' }); * console.log(video.title); * * @param params - The parameters required to fetch the YouTube video. * ...Similar examples would be beneficial for the channel and playlist methods.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/services/youtube.ts(3 hunks)
🧰 Additional context used
🧬 Code Definitions (1)
src/services/youtube.ts (1)
src/types.ts (4) (4)
YoutubeVideo(80-95)YoutubeChannel(97-106)YoutubePlaylist(108-116)SupadataError(53-78)
🔇 Additional comments (5)
src/services/youtube.ts (5)
2-9: The import statement properly includes all necessary typesThe import statement has been correctly updated to include the new types needed for the added YouTube functionality, maintaining proper organization.
31-33: The VideoParams interface looks goodThis interface correctly defines the required parameter for identifying a YouTube video.
79-88: Video method looks correct and well-documentedThe implementation properly fetches video data from the API endpoint with appropriate parameter typing and return type.
90-117: Channel method with nested videos method looks goodThe Object.assign pattern creates a nice API for fetching channel information and videos. The implementation correctly validates inputs before making API calls.
119-146: Playlist method with nested videos method is well-implementedSimilar to the channel method, this creates a clean API for fetching playlist information and videos. The implementation has proper parameter validation and follows consistent patterns.
feat(youtube): update example fix(youtube): fix small typos appointed by CodeRabbit fix(youtube): fix validateLimit validation
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
src/types.ts (2)
85-88: Add missing semicolon after object definition.There's a missing semicolon after the channel object definition, which should be added for consistency with the rest of the codebase.
channel: { id: string; name: string; - } + };
100-100: Add missing semicolon after property.There's a missing semicolon after the handle property, which should be added for consistency with the rest of the codebase.
- handle: string + handle: string;src/services/youtube.ts (1)
150-152: Consider simplifying the null/undefined check.The current check is more verbose than necessary and could be simplified.
- if ( - params.limit != undefined && - params.limit != null && - (params.limit < 1 || params.limit > 5000) - ) { + if ( + params.limit !== undefined && + (params.limit < 1 || params.limit > 5000) + ) {Or even more concisely:
- if ( - params.limit != undefined && - params.limit != null && - (params.limit < 1 || params.limit > 5000) - ) { + if (params.limit !== undefined && (params.limit < 1 || params.limit > 5000)) {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (5)
README.md(2 hunks)example/index.ts(1 hunks)src/__tests__/supadata.test.ts(2 hunks)src/services/youtube.ts(3 hunks)src/types.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- example/index.ts
- README.md
🧰 Additional context used
🧬 Code Definitions (2)
src/services/youtube.ts (1)
src/types.ts (4) (4)
YoutubeVideo(80-95)YoutubeChannel(97-106)YoutubePlaylist(108-116)SupadataError(53-78)
src/__tests__/supadata.test.ts (1)
src/types.ts (1) (1)
YoutubeVideo(80-95)
🪛 Biome (1.9.4)
src/__tests__/supadata.test.ts
[error] 4-4: Do not shadow the global "Map" property.
Consider renaming this variable. It's easy to confuse the origin of variables when they're named after a known global.
(lint/suspicious/noShadowRestrictedNames)
🔇 Additional comments (15)
src/types.ts (3)
80-95: LGTM: YoutubeVideo interface is well-structured.The YoutubeVideo interface correctly captures all necessary properties for YouTube video metadata. The properties are well-typed and provide a comprehensive representation of video data.
97-106: LGTM: YoutubeChannel interface is well-designed.The YoutubeChannel interface properly defines all essential properties for channel metadata with appropriate types. This provides a clear structure for working with YouTube channel data.
108-116: LGTM: YoutubePlaylist interface is properly defined.The YoutubePlaylist interface captures all relevant properties for YouTube playlists with appropriate types. This provides a clear structure for working with playlist data.
src/__tests__/supadata.test.ts (7)
80-114: LGTM: Video test is thorough and well-structured.The test correctly validates the retrieval of video information, mocks API responses, and verifies the correct API endpoint is called with proper parameters. The mock response matches the YoutubeVideo interface structure.
116-131: LGTM: Error handling test for non-existent videos is properly implemented.The test correctly validates the error handling when a video doesn't exist, ensuring the API correctly rejects with the expected error message.
133-166: LGTM: Channel test is comprehensive.The test properly validates the retrieval of channel information, mocks API responses, and verifies the correct API endpoint is called with proper parameters. The mock response structure aligns with the YoutubeChannel interface.
168-184: LGTM: Error handling test for non-existent channels is properly implemented.The test correctly validates the error handling when a channel doesn't exist, ensuring the API correctly rejects with the expected error message.
186-270: LGTM: Channel videos tests are well-implemented.These tests thoroughly validate:
- Retrieving channel videos with and without limits
- Error handling for non-existent channels
- Validation of invalid limit values (0, -1, 10000)
Good job covering all these edge cases.
272-318: LGTM: Playlist tests are comprehensive.The tests properly validate the retrieval of playlist information and error handling for non-existent playlists. The mock response structure is appropriate.
320-404: LGTM: Playlist videos tests are thorough.These tests comprehensively validate:
- Retrieving playlist videos with and without limits
- Error handling for non-existent playlists
- Validation of invalid limit values (0, -1, 10000)
Excellent test coverage of both normal operation and edge cases.
src/services/youtube.ts (5)
31-45: LGTM: Interface hierarchy is well-designed.The interfaces for parameters are logically structured with good use of inheritance:
VideoParamsserves as the base interfaceChannelParamsandPlaylistParamsextend it appropriatelyChannelVideosParamsandPlaylistVideosParamsadd limit functionalityThis design promotes code reuse and maintains a consistent parameter structure.
79-88: LGTM: Video method is well-implemented.The
videomethod is properly documented with JSDoc comments and follows the established pattern for API methods. It returns the correct type and properly passes parameters to the base client.
90-117: LGTM: Channel method and nested videos method are well-implemented.The implementation using
Object.assignto create a method with a nested method is elegant. The documentation is thorough, and the parameter validation is properly implemented. Good job!
119-146: LGTM: Playlist method and nested videos method are well-implemented.Similar to the channel implementation, the playlist method is well-structured with proper documentation and parameter validation. The nested videos method follows the same pattern as the channel videos method for consistency.
148-160: LGTM: Limit validation is properly implemented.The private
validateLimitmethod follows good practices:
- It's reused across multiple methods that need limit validation
- It throws a properly structured
SupadataErrorwith clear error messages- It checks for both undefined and null values before validation
This is a clean implementation that enhances code maintainability.
rafalzawadzki
left a comment
There was a problem hiding this comment.
looks good, just one name change please!
src/services/youtube.ts
Outdated
| export interface VideoParams { | ||
| id: string; | ||
| } | ||
|
|
||
| export interface ChannelParams extends VideoParams {} | ||
|
|
||
| export interface ChannelVideosParams extends ChannelParams { | ||
| limit?: number; | ||
| } | ||
|
|
||
| export interface PlaylistParams extends VideoParams {} | ||
|
|
||
| export interface PlaylistVideosParams extends PlaylistParams { | ||
| limit?: number; | ||
| } |
There was a problem hiding this comment.
I find it confusing that eg PlaylistParams extends VideoParams.
Can we change VideoParams to something more generic like YoutubeEntityParams, ResourceParams etc?
There was a problem hiding this comment.
@rafalzawadzki changed to ResourceParams, see if it's better, just let me know if you need any other changes.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/services/youtube.ts (1)
144-156: Simplify null/undefined checkThe current null/undefined check can be simplified while maintaining the same functionality.
private validateLimit(params: { limit?: number }) { if ( - params.limit != undefined && - params.limit != null && + params.limit != null && (params.limit < 1 || params.limit > 5000) ) { throw new SupadataError({ error: 'invalid-request', message: 'Invalid limit.', details: 'The limit must be between 1 and 5000.', }); } }Note: In JavaScript, using
!= nullchecks for bothnullandundefined, so the separate check forundefinedis redundant.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/services/youtube.ts(3 hunks)
🧰 Additional context used
🧬 Code Definitions (1)
src/services/youtube.ts (1)
src/types.ts (4) (4)
YoutubeVideo(80-95)YoutubeChannel(97-106)YoutubePlaylist(108-116)SupadataError(53-78)
🔇 Additional comments (4)
src/services/youtube.ts (4)
31-41: Interface design looks goodThe
ResourceParamsinterface serves as a clean base interface for YouTube resources, making the code more maintainable. This effectively addresses the past review comment about having a more generic params interface instead of extendingVideoParams.
75-84: Proper implementation of video retrieval endpointThe
videomethod is well-implemented with proper typing and documentation. The parameters and return types match the expectedYoutubeVideointerface defined in the types file.
86-113: Good use of Object.assign for nested method patternUsing
Object.assignto create a function with a nestedvideosmethod is an interesting pattern that provides a clean API. The implementation properly validates parameters and follows the same pattern as other endpoints.
115-142: Consistent implementation for playlist endpointThe playlist implementation follows the same pattern as the channel endpoint, providing consistency in the API design. Good reuse of the
validateLimitmethod for parameter validation.
Fixes #4
/claim #4
Requirements
Implemented the support for all required endpoints, aligned with patterns in this library and the python one. Implemented tests using the content from some real requests I made and all the tests are passing.
When installing the
devDependencies, I also received some warnings due to old versions of packages inpackage-lock.json:So, I also updated the versions with
npm audit fixto handle those warnings.Summary by CodeRabbit