initial commit of Blog content: "using step decorator for bedrock fine tuning" (https://sim.amazon.com/issues/ML-16440) #4657
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
zhaoqizqwang
approved these changes
Jul 8, 2024
zhaoqizqwang
added a commit
that referenced
this pull request
Jul 26, 2024
* Created archived folder and moved all workshop notebooks with 0 views to archived * Moved 10 notebooks with 0 views into archived/notebooks * Moved 13 0 view notebooks to archived * Deleted 17 duplicate notebooks * Deleted 17 duplicate notebooks (#4685) * Update SMP v2 notebooks to use latest PyTorch 2.3.1, TSM 2.4.0 release (#4678) * Update SMP v2 notebooks to use latest PT2.3.1-TSM2.4.0 release. * Update SMP v2 shared_scripts * Update minimum sagemaker pysdk version to 2.224 * Updated README, removed broken links and fixed markdown (#4687) * Parsash2 patch 1 (#4690) * tutorials-after-initial-feedback Added descriptive text to make the notebooks stand on their own. * move athena notebook into dedicated folder * renamed athena end2end notebooks * moved pyspark notebook into dedicated directory * minor change: consistent directory naming convention * Added overview, headers, and explantory text Tested the notebook end to end. Added more context for processing jobs and cleaning up. The output is visible in the cells. * Added overview, headers, explanatory text Also added troubleshooting note from further testing. * fix directory locations for new notebooks * clear notebook outputs * added integration for ci test results * updated formatting with black-nb * update athena notebook: fix parse predictions * fixed ci integration for pyspark-etl-training notebook --------- Co-authored-by: Janosch Woschitz <jwos@amazon.de> * Archived remaining geospatial example notebooks * Removed geospatial from README.md * New Folder Structure Implementation - Archived remaining geospatial example notebooks (#4691) * Archived remaining geospatial example notebooks * Removed geospatial from README.md * Archived remaining workshop notebooks * Archived outdated example notebooks between 1-90 views * MLflow setup (#4689) * Add SageMaker MLflow examples * Add badges * Add MLflow setup notebook; upgrade SageMaker Python SDK for deployment notebook * Linting * More linting changes --------- Co-authored-by: Bobby Lindsey <bwlind@amazon.com> * feat: Model monitor json support for Explainability and Bias (#4696) * initial commit of Blog content: "using step decorator for bedrock fine tuning" (https://sim.amazon.com/issues/ML-16440) (#4657) * initial commit of using step decorator for bedrock fine tuning * ran black command on the notebook * Added CI badges * Added CI badges * fixed typo in notebook title --------- Co-authored-by: Ashish Rawat <rawataws@amazon.com> Co-authored-by: Zhaoqi <jzhaoqwa@amazon.com> * New folder structure (#4694) * Deleted 17 duplicate notebooks (#4685) * Updated README, removed broken links and fixed markdown (#4687) * New Folder Structure Implementation - Archived remaining geospatial example notebooks (#4691) * Archived remaining geospatial example notebooks * Removed geospatial from README.md * Archived remaining workshop notebooks (#4692) * Archived outdated example notebooks between 1-90 views (#4693) --------- Co-authored-by: jsmul <jsmul@amazon.com> * Revert "New folder structure (#4694)" (#4701) This reverts commit 970d88e due to broken blog links * archived 17 notebookswith outdated/redundant funtionality * adding notebook for forecast to canvas workshop (#4704) * adding notebook for forecast to canvas workshop * formatting the notebook using black * Adds notebook for deploying and monitoring llm on sagemaker usin fmeval for evaluation (#4705) Co-authored-by: Brent Friedman <brentfr@amazon.com> * archived 20 notebooks with outdated/redundant functionality * archived 20 notebooks with outdated/redundant funtionality * archived 20 notebooks with outdated/redundant funtionality * archived 21 notebooks with outdated/redundant funtionality * archived 19 notebooks with outdated/redundant funtionality * restored pytorch_multi_model_endpoint back from archived * removed redundant notebooks folder from archived - all notebooks now directly in archived * added new folders for new file structure * added gitkeep files to show folders on github * archived one notebook that was missed * introducing new file structure - part 1 * Update README.md * moved unsorted file back to top level to maintain links * archived recently marked, and removed folder names from file names * new file structure: renamed and moved all evaluated notebooks as of 26 july * new file structure: organized new files and files that still need to be evaluated * Update README.md --------- Co-authored-by: Victor Zhu <viczhu@amazon.com> Co-authored-by: parsash2 <60193914+parsash2@users.noreply.github.com> Co-authored-by: Janosch Woschitz <jwos@amazon.de> Co-authored-by: Bobby Lindsey <bobbywlindsey@users.noreply.github.com> Co-authored-by: Bobby Lindsey <bwlind@amazon.com> Co-authored-by: zicanl-amazon <115581573+zicanl-amazon@users.noreply.github.com> Co-authored-by: ashrawat <ashrawat_atl@yahoo.com> Co-authored-by: Ashish Rawat <rawataws@amazon.com> Co-authored-by: Zhaoqi <jzhaoqwa@amazon.com> Co-authored-by: pro-biswa <bisu@amazon.com> Co-authored-by: brentfriedman725 <97409987+brentfriedman725@users.noreply.github.com> Co-authored-by: Brent Friedman <brentfr@amazon.com>
zhaoqizqwang
added a commit
that referenced
this pull request
Aug 6, 2024
* Update SMP v2 notebooks to use latest PyTorch 2.3.1, TSM 2.4.0 release (#4678) * Update SMP v2 notebooks to use latest PT2.3.1-TSM2.4.0 release. * Update SMP v2 shared_scripts * Update minimum sagemaker pysdk version to 2.224 * Parsash2 patch 1 (#4690) * tutorials-after-initial-feedback Added descriptive text to make the notebooks stand on their own. * move athena notebook into dedicated folder * renamed athena end2end notebooks * moved pyspark notebook into dedicated directory * minor change: consistent directory naming convention * Added overview, headers, and explantory text Tested the notebook end to end. Added more context for processing jobs and cleaning up. The output is visible in the cells. * Added overview, headers, explanatory text Also added troubleshooting note from further testing. * fix directory locations for new notebooks * clear notebook outputs * added integration for ci test results * updated formatting with black-nb * update athena notebook: fix parse predictions * fixed ci integration for pyspark-etl-training notebook --------- Co-authored-by: Janosch Woschitz <jwos@amazon.de> * MLflow setup (#4689) * Add SageMaker MLflow examples * Add badges * Add MLflow setup notebook; upgrade SageMaker Python SDK for deployment notebook * Linting * More linting changes --------- Co-authored-by: Bobby Lindsey <bwlind@amazon.com> * feat: Model monitor json support for Explainability and Bias (#4696) * initial commit of Blog content: "using step decorator for bedrock fine tuning" (https://sim.amazon.com/issues/ML-16440) (#4657) * initial commit of using step decorator for bedrock fine tuning * ran black command on the notebook * Added CI badges * Added CI badges * fixed typo in notebook title --------- Co-authored-by: Ashish Rawat <rawataws@amazon.com> Co-authored-by: Zhaoqi <jzhaoqwa@amazon.com> * New folder structure (#4694) * Deleted 17 duplicate notebooks (#4685) * Updated README, removed broken links and fixed markdown (#4687) * New Folder Structure Implementation - Archived remaining geospatial example notebooks (#4691) * Archived remaining geospatial example notebooks * Removed geospatial from README.md * Archived remaining workshop notebooks (#4692) * Archived outdated example notebooks between 1-90 views (#4693) --------- Co-authored-by: jsmul <jsmul@amazon.com> * Revert "New folder structure (#4694)" (#4701) This reverts commit 970d88e due to broken blog links * adding notebook for forecast to canvas workshop (#4704) * adding notebook for forecast to canvas workshop * formatting the notebook using black * Adds notebook for deploying and monitoring llm on sagemaker usin fmeval for evaluation (#4705) Co-authored-by: Brent Friedman <brentfr@amazon.com> * Update athena_ml_workflow_end_to_end.ipynb (#4702) Fixed bucket names and external links. No change to underlying code or formatting. Co-authored-by: sage-maker <parknate@amazon.com> * SageMaker FasterAutoscaling Llama3-8B TGI, real-time endpoints (#4712) * SageMaker FasterAutoscaling Llama3-8B TGI, real-time endpoints * Moved trigger autoscaling to shell script. Removed shell=True in subprocess.Popen --------- Co-authored-by: Aditi Sharma <165942273+Aditi2424@users.noreply.github.com> * Removed bug where automatic permission attachment errors (#4714) * new file structure: re-added pull request template and files needed for rtd build * SageMaker pipeline step decorator blog (#4715) * initial commit of using step decorator for bedrock fine tuning * ran black command on the notebook * Added CI badges * Added CI badges * fixed typo in notebook title * added comments and reviewer feedback --------- Co-authored-by: Ashish Rawat <rawataws@amazon.com> Co-authored-by: Zhaoqi <jzhaoqwa@amazon.com> * new file structure: re-added RTD files, finished archiving/organizing, added READMEs to folders * new file structure: updated index.rst with new file structure / archived one notebook, renamed one notebook * new file structure: updated contributing.md * LLM Monitoring Multiple Evals (#4718) * Removed bug where automatic permission attachment errors * Adds notebook for monitoring llm with multiple eval libraries --------- Co-authored-by: Brent Friedman <brentfr@amazon.com> Co-authored-by: nileshvd <113946607+nileshvd@users.noreply.github.com> * new file structure: updated badging path for end-to-end examples * new file structure: updated badging path for prepare data notebooks * new file structure: updated badging path for build and train models folder * new file structure: updated badging path for deploy and monitor folder * new file structure: updated badging path for generative ai folder * new file structure: updated badging path for ml ops folder * new file structure: updated badging path for responsible ai folder * new file structure: updated folder READMEs, and added several notebook titles * new folder structure: updated pull request template * new file structure: updated readme with announcement, added new notebook naming excel sheet --------- Co-authored-by: Victor Zhu <viczhu@amazon.com> Co-authored-by: parsash2 <60193914+parsash2@users.noreply.github.com> Co-authored-by: Janosch Woschitz <jwos@amazon.de> Co-authored-by: Bobby Lindsey <bobbywlindsey@users.noreply.github.com> Co-authored-by: Bobby Lindsey <bwlind@amazon.com> Co-authored-by: zicanl-amazon <115581573+zicanl-amazon@users.noreply.github.com> Co-authored-by: ashrawat <ashrawat_atl@yahoo.com> Co-authored-by: Ashish Rawat <rawataws@amazon.com> Co-authored-by: Zhaoqi <jzhaoqwa@amazon.com> Co-authored-by: pro-biswa <bisu@amazon.com> Co-authored-by: brentfriedman725 <97409987+brentfriedman725@users.noreply.github.com> Co-authored-by: Brent Friedman <brentfr@amazon.com> Co-authored-by: sage-maker <parknate@amazon.com> Co-authored-by: Praveen Chamarthi <118387029+pchamart@users.noreply.github.com> Co-authored-by: Aditi Sharma <165942273+Aditi2424@users.noreply.github.com> Co-authored-by: nileshvd <113946607+nileshvd@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available: ML-16440
Description of changes:
Added 3 files including the Jupyter notebook containing code for the blog
Testing done: Yes
Merge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.black-nb -l 100 {path}/{notebook-name}.ipynbBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.