Quickly Deploying Gradio on AWS
You’ve built a demo for your machine learning model with Gradio. Now, deploy it with a permanent link on an AWS instance
If you’ve built a machine learning model that works well, you might want to demo it, so that others can try it out. The Gradio library makes it really easy to create a shareable GUI for your model, as long as the model is running on your local computer. But what if you need a long-term hosted solution?
In this tutorial, I’m going to show you, step by step, how to create and deploy your machine learning model and UI on an AWS EC2 instance. This is useful if you need a permanent link for your model interface to share within your company or organization.
I’m going to use this image denoising model and the interface I created for it using Gradio as an example:
It’s a quick process to get your model up too, so let’s get started!
The Gradio Preliminaries
I’m assuming that you’re already somewhat familiar with the Gradio library (https://github.com/gradio-app/gradio). If not, this quick start is a good place to get started. As a refresher,
gradio is a Python library that lets you create UIs around your machine learning model by specifying 3 things: a Python function, input component, and an output component.
The AWS EC2 Preliminaries
I’m also assuming that you already have an EC2 instance running Linux and Python3 and you know how to SSH into the machine to run commands on it. If not, here’s a good tutorial to get started with an EC2 instance and a tutorial to install Python3 on your instance.
Step 1: Create a Gradio App File for Your Model
The first step is to create a file that launches the Gradio GUI for your model. Give it a simple name like
demo.py. This file should:
- Load the model
- Define a prediction function using the model
- Launch a Gradio interface with the prediction function and appropriate UI components
Here’s an example such file from the Noise2Same repo, which contains a state-of-the-art image denoising model:
You should run this file locally (for example, by running
python3 demo.py) to make sure that the Gradio app launches successfully.
Step 2: Make the 1 change needed to run Gradio on an EC2 instance
You’re going to need to make just one change so that the Gradio app runs successfully on the EC2 instance. That’s adding a
server_name argument to the
Interface class and setting it to be
0.0.0.0 . So the modified code becomes:
This allows your Gradio app to be accessible to users from other computers (i.e. across the web).
Step 3: Publish your model as a GitHub repo & include a requirements.txt file
Once your code is ready, make a GitHub repository for your account with all of the necessary files, including the Gradio app file that you just created. You can use a different version control system if you’d like — we just need to get the code onto the EC2 instance.
We also need to make sure that we designate all of the Python libraries that we will need as dependencies, along with their versions if necessary. So create a
requirements.txt file in the root of your repo. Don’t forget
gradio. In the case of the Noise2Same repo, this is what the requirements look like:
Once you have added these files to your GitHub project, push your updated repo to GitHub.
Step 4: SSH into your EC2 instance and clone the repo & install the dependencies
SSH into your EC2 instance, and then do the following:
- (Recommended) create a new tmux session by typing
tmux(this will allow the app to continue running even after you end your connection)
- In the tmux session, clone the repository for your Gradio application
pip3 install -r requirements.txtto install all of the requirements for your Python project
Step 5: Run the Gradio app
Finally, run your Gradio application! For example, if your Gradio file was called
demo.py , then simply run the following command:
Step 6: Allow web traffic to the port
Now, if everything ran smoothly, you should get something like the following output:
This tells you that in our case, the Gradio app is running on port
7860. We now need to allow people to visit that port on our machine. To do that, we edit the security group associated with our instance:
- Navigate to Network & Security settings on the left-hand navigation, and click Security Groups.
- Find the security group connected to your instance
- Choose Inbound rules and click Edit inbound rules
- Click the Add rule button
- For the new rule, under Type, select “Custom TCP.” In Port Range, type the port number (in our case
- In the Source field, enter the range of IP addresses that should be able to access the machine. If you select My IP only your current IP address (i.e. your machine) will be able to visit the machine. If you select Anywhere then anyone across the web will be able to access the interface. You can specify the range of IP addresses as well if you’d like finer control over access.
- Then click Save rules
And that’s it! Your Gradio interface should now be accessibly by simply typing the public IP address of your machine followed by colon, then the port number. In my case, that was:
Hope that helps! Although this solution lets you quickly deploy a Gradio interface, this is not meant to be a production-ready deployment! For that, you’ll want to read more detailed guides, or talk to a Gradio expert.
If you’d like to consult with a Gradio expert in launching your own Gradio interface or learning about advanced features like authentication, security, and traffic handling, then send us a message at email@example.com!