| Beware that the custom tool should follow the exact same API | |
| as the overwritten tool in this case, or you should adapt the prompt template to make sure all examples using that | |
| tool are updated. | |
| The upscaler tool was given the name image_upscaler which is not yet present in the default toolbox and is therefore simply added to the list of tools. | |
| You can always have a look at the toolbox that is currently available to the agent via the agent.toolbox attribute: | |
| py | |
| print("\n".join([f"- {a}" for a in agent.toolbox.keys()])) | |
| text | |
| - document_qa | |
| - image_captioner | |
| - image_qa | |
| - image_segmenter | |
| - transcriber | |
| - summarizer | |
| - text_classifier | |
| - text_qa | |
| - text_reader | |
| - translator | |
| - image_transformer | |
| - text_downloader | |
| - image_generator | |
| - video_generator | |
| - image_upscaler | |
| Note how image_upscaler is now part of the agents' toolbox. | |
| Let's now try out the new tools! We will re-use the image we generated in Transformers Agents Quickstart. | |
| from diffusers.utils import load_image | |
| image = load_image( | |
| "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/rivers_and_lakes.png" | |
| ) | |
| Let's transform the image into a beautiful winter landscape: | |
| py | |
| image = agent.run("Transform the image: 'A frozen lake and snowy forest'", image=image) | |
| ``text | |
| ==Explanation from the agent== | |
| I will use the following tool:image_transformer` to transform the image. | |
| ==Code generated by the agent== | |
| image = image_transformer(image, prompt="A frozen lake and snowy forest") | |
| The new image processing tool is based on ControlNet which can make very strong modifications to the image. | |
| By default the image processing tool returns an image of size 512x512 pixels. |