AI Apps in a Flash with Gradio's Reload Mode
โข
23
Freddy Boulton
freddyaboulton
AI & ML interests
None yet
Articles
Organizations
freddyaboulton's activity
posted
an
update
5 months ago
Post
1221
@dwancin
Can you please reset your toggle component's space? It's stuck for some reason. Happy to help
dwancin/gradio_toggle
dwancin/gradio_toggle
reacted to
m-ric's
post with ๐คฏ๐ค๐ฅ๐
7 months ago
Post
2182
๐ก๐ฒ๐ ๐ฆ๐ฝ๐ฎ๐ฐ๐ฒ: ๐ผ๐ ๐๐ง๐๐ซ๐๐ก ๐ฅ๐ก๐๐ฃ๐ฃ๐๐ง ๐บ๏ธ๐๏ธ Plan your next vacation in a few minutes!
I wanted to try out if a powerful LLM like Mixtral-8x7b had geographical reasoning capabilities.
So I built a small space that prompts the LLM to provide a JSON list of places based on a user input.
And the result was impressive! ๐คฏ
โ ๐๐ ๐๐ฒ๐ฒ๐บ๐ ๐น๐ถ๐ธ๐ฒ ๐ ๐ถ๐ ๐๐ฟ๐ฎ๐น ๐ต๐ฎ๐ ๐ฎ ๐ด๐ฟ๐ฎ๐๐ฝ ๐ผ๐ณ ๐ด๐ฒ๐ผ๐ด๐ฟ๐ฎ๐ฝ๐ต๐ถ๐ฐ๐ฎ๐น ๐ฐ๐ผ๐ป๐ฐ๐ฒ๐ฝ๐๐ ๐น๐ถ๐ธ๐ฒ ๐ก๐ผ๐ฟ๐๐ต - ๐ฆ๐ผ๐๐๐ต, ๐ผ๐ฟ ๐๐ฝ๐ฎ๐๐ถ๐ฎ๐น ๐ฎ๐น๐ถ๐ด๐ป๐บ๐ฒ๐ป๐.๐งญ Not just describing these concepts, but really applying them in practice, for instance to successfully answer "give me 4 European cities that are aligned on the map". This is a ๐ป๐ถ๐ฐ๐ฒ ๐ฒ๐ ๐ฎ๐บ๐ฝ๐น๐ฒ ๐ผ๐ณ ๐ฎ๐ป ๐ฒ๐บ๐ฒ๐ฟ๐ด๐ฒ๐ป๐ ๐ฐ๐ฎ๐ฝ๐ฎ๐ฏ๐ถ๐น๐ถ๐๐, since nothing in the LLM's training data should prepare it for this specific task.
Anyway, I added API calls and a nice visualization on top of the LLM, streaming output, caching for the answers and locations... and ta-da! โจ I got the ๐๐ ๐ง๐ฟ๐ฎ๐๐ฒ๐น ๐ฃ๐น๐ฎ๐ป๐ป๐ฒ๐ฟ.
๐๐ค๐ช ๐๐๐ฃ ๐๐๐จ๐๐ง๐๐๐ ๐๐ฉ ๐ฎ๐ค๐ช๐ง ๐ฉ๐ง๐๐ฅ, ๐๐ฃ๐ ๐๐ฉ ๐ฌ๐๐ก๐ก ๐๐ค๐ข๐ ๐ช๐ฅ ๐ฌ๐๐ฉ๐ ๐ฃ๐๐๐ ๐๐ฃ๐ ๐๐ค๐ฃ๐ซ๐๐ฃ๐๐๐ฃ๐ฉ ๐ก๐ค๐๐๐ฉ๐๐ค๐ฃ๐จ!
๐๐ง๐ฎ ๐๐ฉ ๐๐๐ง๐ ๐ m-ric/ai-travel-planner
Thank you @freddyaboulton for the ๐๐๐๐๐๐_๐๐๐๐๐๐ component, and @clem , @pngwn , @abidlabs for your ideas and support!
I wanted to try out if a powerful LLM like Mixtral-8x7b had geographical reasoning capabilities.
So I built a small space that prompts the LLM to provide a JSON list of places based on a user input.
And the result was impressive! ๐คฏ
โ ๐๐ ๐๐ฒ๐ฒ๐บ๐ ๐น๐ถ๐ธ๐ฒ ๐ ๐ถ๐ ๐๐ฟ๐ฎ๐น ๐ต๐ฎ๐ ๐ฎ ๐ด๐ฟ๐ฎ๐๐ฝ ๐ผ๐ณ ๐ด๐ฒ๐ผ๐ด๐ฟ๐ฎ๐ฝ๐ต๐ถ๐ฐ๐ฎ๐น ๐ฐ๐ผ๐ป๐ฐ๐ฒ๐ฝ๐๐ ๐น๐ถ๐ธ๐ฒ ๐ก๐ผ๐ฟ๐๐ต - ๐ฆ๐ผ๐๐๐ต, ๐ผ๐ฟ ๐๐ฝ๐ฎ๐๐ถ๐ฎ๐น ๐ฎ๐น๐ถ๐ด๐ป๐บ๐ฒ๐ป๐.๐งญ Not just describing these concepts, but really applying them in practice, for instance to successfully answer "give me 4 European cities that are aligned on the map". This is a ๐ป๐ถ๐ฐ๐ฒ ๐ฒ๐ ๐ฎ๐บ๐ฝ๐น๐ฒ ๐ผ๐ณ ๐ฎ๐ป ๐ฒ๐บ๐ฒ๐ฟ๐ด๐ฒ๐ป๐ ๐ฐ๐ฎ๐ฝ๐ฎ๐ฏ๐ถ๐น๐ถ๐๐, since nothing in the LLM's training data should prepare it for this specific task.
Anyway, I added API calls and a nice visualization on top of the LLM, streaming output, caching for the answers and locations... and ta-da! โจ I got the ๐๐ ๐ง๐ฟ๐ฎ๐๐ฒ๐น ๐ฃ๐น๐ฎ๐ป๐ป๐ฒ๐ฟ.
๐๐ค๐ช ๐๐๐ฃ ๐๐๐จ๐๐ง๐๐๐ ๐๐ฉ ๐ฎ๐ค๐ช๐ง ๐ฉ๐ง๐๐ฅ, ๐๐ฃ๐ ๐๐ฉ ๐ฌ๐๐ก๐ก ๐๐ค๐ข๐ ๐ช๐ฅ ๐ฌ๐๐ฉ๐ ๐ฃ๐๐๐ ๐๐ฃ๐ ๐๐ค๐ฃ๐ซ๐๐ฃ๐๐๐ฃ๐ฉ ๐ก๐ค๐๐๐ฉ๐๐ค๐ฃ๐จ!
๐๐ง๐ฎ ๐๐ฉ ๐๐๐ง๐ ๐ m-ric/ai-travel-planner
Thank you @freddyaboulton for the ๐๐๐๐๐๐_๐๐๐๐๐๐ component, and @clem , @pngwn , @abidlabs for your ideas and support!
- 1 reply
replied to
their
post
7 months ago
Here is a demo of the API recorder โจ
posted
an
update
7 months ago
Post
3394
We just released gradio version 4.26.0 ! We *highly* recommend you upgrade your apps to this version to bring in these nice changes:
๐ฅ Introducing the API recorder. Any gradio app running 4.26.0 and above will have an "API Recorder" that will record your interactions with the app and auto-generate the corresponding python or js code needed to recreate those actions programmatically. It's very neat!
๐ Enhanced markdown rendering in gr.Chatbot
๐ข Fix for slow load times on spaces as well as the UI locking up on rapid generations
See the full changelog of goodies here: https://www.gradio.app/changelog#4-26-0
๐ฅ Introducing the API recorder. Any gradio app running 4.26.0 and above will have an "API Recorder" that will record your interactions with the app and auto-generate the corresponding python or js code needed to recreate those actions programmatically. It's very neat!
๐ Enhanced markdown rendering in gr.Chatbot
๐ข Fix for slow load times on spaces as well as the UI locking up on rapid generations
See the full changelog of goodies here: https://www.gradio.app/changelog#4-26-0
-
- 1 reply
reacted to
julien-c's
post with โค๏ธ
7 months ago
Post
6565
text-generation-inference (TGI) is now fully open-source again!
Along with text-embeddings-inference.
We just switched both of those repos' license back to Apache 2. ๐ฅ
Along with text-embeddings-inference.
We just switched both of those repos' license back to Apache 2. ๐ฅ
posted
an
update
7 months ago
Post
2413
Gradio 4.25.0 is out with some nice improvements and bug fixes!
๐งน Automatic deletion of gr.State variables stored in the server. Never run out of RAM again. Also adds an unload event you can run when a user closes their browser tab.
๐ด Lazy example caching. You can set cache_examples="lazy" to cache examples when they're first requested as opposed to before the server launches. This can cut down the server's start-up time drastically.
๐ Fixes a bug with streaming audio outputs
๐ค Improvements to gr.ChatInterface like pasting images directly from the clipboard.
See the rest of the changelog here: https://www.gradio.app/changelog#4-25-0
๐งน Automatic deletion of gr.State variables stored in the server. Never run out of RAM again. Also adds an unload event you can run when a user closes their browser tab.
๐ด Lazy example caching. You can set cache_examples="lazy" to cache examples when they're first requested as opposed to before the server launches. This can cut down the server's start-up time drastically.
๐ Fixes a bug with streaming audio outputs
๐ค Improvements to gr.ChatInterface like pasting images directly from the clipboard.
See the rest of the changelog here: https://www.gradio.app/changelog#4-25-0
replied to
their
post
8 months ago
Not with the format parameter. That only applies if you return a numpy or PIL image from your prediction function. If you want to control how files are saved to jpeg, you can do that within your prediction function and gradio will not modify it further.
posted
an
update
8 months ago
Post
1663
Tips for saving disk space with Gradio ๐พ
Try these out with gradio 4.22.0 ! Code snippet attached.
1. Set delete_cache. The delete_cache parameter will periodically delete files from gradio's cache that are older than a given age. Setting it will also delete all files created by that app when the app shuts down. It is a tuple of two ints, (frequency, age) expressed in seconds. So delete_cache=(3600, 3600), will delete files older than an hour every hour.
2. Use static files. Static files are not copied to the cache and are instead served directly to users of your app. This is useful for components displaying a lot of content that won't change, like a gallery with hundreds of images.
3. Set format="jpeg" for images and galleries. JPEGs take up less disk space than PNGs. This can also speed up the speed of your prediction function as they will be written to the cache faster.
Try these out with gradio 4.22.0 ! Code snippet attached.
1. Set delete_cache. The delete_cache parameter will periodically delete files from gradio's cache that are older than a given age. Setting it will also delete all files created by that app when the app shuts down. It is a tuple of two ints, (frequency, age) expressed in seconds. So delete_cache=(3600, 3600), will delete files older than an hour every hour.
2. Use static files. Static files are not copied to the cache and are instead served directly to users of your app. This is useful for components displaying a lot of content that won't change, like a gallery with hundreds of images.
3. Set format="jpeg" for images and galleries. JPEGs take up less disk space than PNGs. This can also speed up the speed of your prediction function as they will be written to the cache faster.
-
- 3 replies
reacted to
jimfan's
post with ๐
9 months ago
Post
NVIDIA released Chat with RTX. Why is it significant? It's NVIDIA's first step towards the vision of "LLM as Operating System" - a locally running, heavily optimized AI assistant that deeply integrates with your file systems, features retrieval as the first-class citizen, preserves privacy, and leverages your desktop gaming GPUs to the full.
A web-hosted version will have a hard time matching the same functionality. NVIDIA is going local even before OpenAI.
https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generative-ai/
A web-hosted version will have a hard time matching the same functionality. NVIDIA is going local even before OpenAI.
https://www.nvidia.com/en-us/ai-on-rtx/chat-with-rtx-generative-ai/
reacted to
abidlabs's
post with โค๏ธ
10 months ago
Post
Just out: new custom Gradio component specifically designed for code completion models ๐ฅ
- 1 reply
reacted to
gsarti's
post with โค๏ธ๐ค
10 months ago
Post
Are you working with users and would like to let them edit a text containing highlights? Check out my new Gradio component
The component can be installed with
Demo: gsarti/gradio_highlightedtextbox
HighlightedTextbox! ๐คThe component can be installed with
pip install gradio_highlightedtextbox and used seamlessly with other native components in Gradio interfaces. It supports texts with multiple tags and provides some reasonable UX behaviors (tags disappear when an edit is performed inside them). It should be great for UIs to study user editing of annotated LM generations/translations!Demo: gsarti/gradio_highlightedtextbox
-
- 1 reply
reacted to
abidlabs's
post with โค๏ธ
10 months ago
Post
Gradio 4.16 introduces a new flow: you can hide/show Tabs or make them interactive/non-interactive.
Really nice for multi-step machine learning ademos โก๏ธ
Really nice for multi-step machine learning ademos โก๏ธ
-
- 6 replies
posted
an
update
10 months ago
Post
New in Gradio 4.16.0 - Galleries as Input ๐ผ๏ธ
Now your users can upload multiple images as input to your AI application and view them in a slick gallery!
Attached is a demo of how this new feature can be used in a photomaker-type application: TencentARC/PhotoMaker
Shout out @abidlabs and @akhaliq who proposed this feature after seeing some of the workarounds gradio developers were using in the wild to upload multiple images.
The gradio team works hard to stay up to date with the latest trends in AI! If there's something missing from the library, file an issue on github! https://github.com/gradio-app/gradio/issues
Now your users can upload multiple images as input to your AI application and view them in a slick gallery!
Attached is a demo of how this new feature can be used in a photomaker-type application: TencentARC/PhotoMaker
Shout out @abidlabs and @akhaliq who proposed this feature after seeing some of the workarounds gradio developers were using in the wild to upload multiple images.
The gradio team works hard to stay up to date with the latest trends in AI! If there's something missing from the library, file an issue on github! https://github.com/gradio-app/gradio/issues
- 2 replies
reacted to
abidlabs's
post with โค๏ธ
10 months ago
Post
โจ Excited to release
๐ปโโ๏ธ Native support for Polars Dataframe
๐ผ๏ธ Gallery component can be used as an input
โก Much faster streaming for low-latency chatbots
๐ Auto generated docs for custom components
... and much more! This is HUGE release, so check out everything else in our changelog: https://github.com/gradio-app/gradio/blob/main/CHANGELOG.md
gradio 4.16. New features include:๐ปโโ๏ธ Native support for Polars Dataframe
๐ผ๏ธ Gallery component can be used as an input
โก Much faster streaming for low-latency chatbots
๐ Auto generated docs for custom components
... and much more! This is HUGE release, so check out everything else in our changelog: https://github.com/gradio-app/gradio/blob/main/CHANGELOG.md
-
- 3 replies
posted
an
update
10 months ago
Post
Making gradio's auto-reload leaner and more robust ๐ช
Auto-reload is one of the coolest (and maybe underrated) features of gradio in my opinion. It automatically detects changes in your python app file and hot swaps your backend and frontend without restarting the server. I may be biased, but that's a way nicer experience than most other reload mode features out there.
This is all possible due to novel use of python's built in importlib module. Since this is a non-standard use of the module, our users have encountered some rough edges in the wild.
I'm happy to announce two improvements that will be out in the next release of gradio. They were both made possible by collaboration in the open source community.
1. First, gradio's reload mode now works with the
2. Second, gradio's reload mode now consumes an order of magnitude less CPU. Depending on how many files are in your source directory, you may see between 2x to 10x less CPU utilization. Thanks to @velaia for filing the issue and @abidlabs for the review!
More improvements to reload mode are planned. Stay tuned!
Auto-reload is one of the coolest (and maybe underrated) features of gradio in my opinion. It automatically detects changes in your python app file and hot swaps your backend and frontend without restarting the server. I may be biased, but that's a way nicer experience than most other reload mode features out there.
This is all possible due to novel use of python's built in importlib module. Since this is a non-standard use of the module, our users have encountered some rough edges in the wild.
I'm happy to announce two improvements that will be out in the next release of gradio. They were both made possible by collaboration in the open source community.
1. First, gradio's reload mode now works with the
python-dotenv library. Getting to the bottom of this one took a lot of sleuthing by our users and the fix got merged and released into the python-dotenv package last night! Thanks to
@theskumar
, maintainer of python-dotenv, for all the help. 2. Second, gradio's reload mode now consumes an order of magnitude less CPU. Depending on how many files are in your source directory, you may see between 2x to 10x less CPU utilization. Thanks to @velaia for filing the issue and @abidlabs for the review!
More improvements to reload mode are planned. Stay tuned!
reacted to
abidlabs's
post with ๐
10 months ago
Post
๐๐ผ๐ ๐๐ฒ ๐บ๐ฎ๐ฑ๐ฒ ๐๐ฟ๐ฎ๐ฑ๐ถ๐ผ ๐ณ๐ฎ๐๐๐ฒ๐ฟ ๐ฏ๐... ๐๐น๐ผ๐๐ถ๐ป๐ด ๐ถ๐ ๐ฑ๐ผ๐๐ป!
About a month ago, @oobabooga (who built the popular text generation webui) reported an interesting issue to the Gradio team. After upgrading to Gradio 4, @oobabooga noticed that chatbots that streamed very quickly had a lag before their text would show up in the Gradio app.
After some investigation, we determined that the Gradio frontend would receive the updates from the backend immediately, but the browser would lag before rendering the changes on the screen. The main difference between Gradio 3 and Gradio 4 was that we migrated the communication protocol between the backend and frontend from Websockets (WS) to Server-Side Events (SSE), but we couldn't figure out why this would affect the browser's ability to render the streaming updates it was receiving.
After diving deep into browsers events, @aliabid94 and @pngwn made a realization: most browsers treat WS events (specifically the
We debated different options, but the solution that we implemented was to introduce throttling: we slowed down how frequently we would push updates to the browser event stack to a maximum rate of 20/sec. Although this seemingly โslowed downโ Gradio streaming, it actually would allow browsers to process updates in real-time and provide a much better experience to end users of Gradio apps.
See the PR here: https://github.com/gradio-app/gradio/pull/7084
Kudos to @aliabid94 and @pngwn for the fix, and to @oobabooga and @pseudotensor for helping us test it out!
About a month ago, @oobabooga (who built the popular text generation webui) reported an interesting issue to the Gradio team. After upgrading to Gradio 4, @oobabooga noticed that chatbots that streamed very quickly had a lag before their text would show up in the Gradio app.
After some investigation, we determined that the Gradio frontend would receive the updates from the backend immediately, but the browser would lag before rendering the changes on the screen. The main difference between Gradio 3 and Gradio 4 was that we migrated the communication protocol between the backend and frontend from Websockets (WS) to Server-Side Events (SSE), but we couldn't figure out why this would affect the browser's ability to render the streaming updates it was receiving.
After diving deep into browsers events, @aliabid94 and @pngwn made a realization: most browsers treat WS events (specifically the
WebSocket.onmessage function) with a lower priority than SSE events (EventSource.onmessage function), which allowed the browser to repaint the window between WS messages. With SSE, the streaming updates would stack up in the browser's event stack and be prioritized over any browser repaint. The browser would eventually clear the stack but it would take some time to go through each update, which produced a lag.We debated different options, but the solution that we implemented was to introduce throttling: we slowed down how frequently we would push updates to the browser event stack to a maximum rate of 20/sec. Although this seemingly โslowed downโ Gradio streaming, it actually would allow browsers to process updates in real-time and provide a much better experience to end users of Gradio apps.
See the PR here: https://github.com/gradio-app/gradio/pull/7084
Kudos to @aliabid94 and @pngwn for the fix, and to @oobabooga and @pseudotensor for helping us test it out!
-
- 3 replies