Implement general awsEndpoint + lambda integration#428
Implement general awsEndpoint + lambda integration#428EikeKohl wants to merge 2 commits intohuggingface:mainfrom
Conversation
|
Hello and thanks for contributing! On paper this looks good to me, I haven't tested yet but what do you think @philschmid ? |
Agree! Looks good, but we need to make sure both ways work and should have some checks that people will not provide, e.g. |
|
Thank you for considering my contribution 🙏. I actually tested it with both, Lambda Function URL and Sagemaker invocation URL and it worked 🥳. @philschmid If P.S. I noticed the workflow failed with a linting error, so I pushed another commit after I ran |
|
Cool, going to close this for now then, as we'll merge the other PR soon. I'll ping you when it's done! |
This small refactoring PR implements a general
awsEndpoint, enabeling the user to not only connect sagemaker endpoints, but also lambda function URLs that require AWS_IAM authentication.I had to implement this change for my personal setup, because upon requesting an LLM generation, the lambda performs additional scaling logic to provision / delete the sagemaker endpoint to save LLM hosting costs. I think that everyone could benefit from this functionality.