Deep Learning Deployment Basics - Neural Network Web Apps

Keras neural network deployment - Access model from Powershell and cURL

In this episode, we're going to explore how we can get predictions from our Keras model in a slightly different way than how we've seen it done in the browser in earlier episodes. Here, we'll be calling our back end Flask web service from both Powershell on Windows, and cURL in a Bash terminal.

You may recall that, in an earlier episode on deploying a Keras model to Flask, I stated that once our model is being hosted by Flask, we'll be able to interact with that model over HTTP regardless of which programming language our front end application is written in.

We've seen all the details for how to build a front end web application with Javascript and HTML that runs in the browser and requests predictions from our model. Now, we're going to get an idea for how easy it is to access our model using other languages and platforms. We're first going to show how this is done using Powershell on a Windows machine, and we'll then see how this is done using cURL in a Bash terminal.

Before we start, note we'll be using the same Flask prediction app that we created earlier, so everything on the back end will be running in the exact same way as before.

Alright, let's get started with PowerShell.

Access neural network app with Powershell

If you're not already familiar, PowerShell comes pre-installed on Windows machines and consists of a command line terminal and an associated PowerShell scripting language. If you're going through this tutorial from a Windows machine, go ahead and type Powershell in the search bar, and you should see it come up.

In Powershell, we're going to access an image file stored on disk and process it to get it in the format that the predict endpoint expects. We'll then make an HTTP post request with the image data, and check out the response we get with the predictions for that image.

So, you see, this is the exact same flow we went through when we developed our web application. We're doing the same thing now, but rather than using Javascript and HTML and interacting with the browser, we're now using Powershell, the scripting language, and interacting with the Powershell console.

First, obtain any image of a cat or dog, and save it on your machine.

Now, in Powershell, we first create a variable called $fileName, and we set that equal to the image file. Note, in the corresponding video, from within Powershell, I'm in the same directory as where my image is saved. If your image is saved elsewhere, then you'll need to provide the full path to the image.

$fileName = 'path\to\image.png'

Next, we create a variable called $bytes, which is assigned the bytes from the image file.

$bytes = [IO.File]::ReadAllBytes($fileName)

We then convert the $bytes into base64, and store the base64 encoded image as this $base64image variable.

$base64Image = [Convert]::ToBase64String($bytes)

We then create a Powershell object called $message, who's key is image and value is the base64 encoded image $base64image.

$message = @{ image = $base64Image }

Since the predict endpoint expects the request to be in JSON, we convert the message to JSON and store it in this $jsonified variable.

$jsonified = ConvertTo-Json $message

We then make the HTTP request using the Invoke-RestMethod command. We supply the type of request, Post, and provide the URL to the predict endpoint along with the $jsonified message, and we store the response for this request in this $response variable.

$response = Invoke-RestMethod -Method Post -Uri "http://10.0.0.4:5000/predict" -Body $jsonified

Lastly, we access prediction from $response, and format it using the format-list command, which will print the formatted predictions to the Powershell console.

$response.prediction | format-list

We then obtain the predictions from our model for this particular image.

cat : 8.905021218E-07
dog : 0.999999

Access neural network app with cURL

Next, we're going to do the exact same thing, but in a Bash terminal with cURL.

So, again, the first thing we do is specify which file we're working with and store it in a variable called fileName.

fileName='path/to/image.png'

Next we convert it to base64, and store that encoding in a variable called base64Image. Note, we skipped the step of converting the data into bytes before we base64 encoded it because the base64 command here can encode the data directly from the image file.

base64Image=$(base64 $fileName)

We then convert it to JSON, again, with the key called image, and the value being the base64Image.

jsonified="{\"image\":\"${base64Image}\"}"

Next, we dump this JSON into a file that we're calling data.json, and we do this because the in-memory image data appeared to be too large to send using the next command, so we're just storing that data in a file that we'll delete after we're done with it.

echo $jsonified >> data.json

Next, we use curl to make our HTTP request. If you're not familiar, curl is a command line tool used for transferring data.

We run the curl command specifying the type of request, POST, the data to send, which is the json data in the data.json file, and the URL to our predict endpoint.

curl -X POST --data @data.json http://10.0.0.4:5000/predict

Afterwards, we get the response printed to the console, as we'd expect.

"prediction": {
    "cat": 8.9050212e-07,
    "dog": 0.99999
}

We can then run rm data.json to delete the file we created.

These examples hopefully allowed you to see how relatively simple it is to make HTTP requests across languages and platforms.

Just with a few lines of code, we were able to send image data and receive predictions with both Powershell and the Bash terminal, and there was no need to change our back end regardless of what we were using as our front end. There is still more to come, so stay tuned, and I'll see ya in the next one!