Automated model serving to mobile devices

  • The need to use the model offline or in low connectivity areas.
  • The need to minimize the amount of data being transferred — perhaps the user lives in a place where data is not cheap nor easily accessible, or the model requires large amounts of data as input.
  • The need to not be limited by network speed, requiring more immediate results
  • Data cleaning: This is the step where incorrect data is filtered out from the input and data is set up to be consumable by the model
  • Feature selection: You might have such a step if the features that you are feeding int your model require some additional computation. You might also have merged the data cleansing and feature selection steps of the pipeline together.
  • Model training: This is where the data is consumed in order to train a model with some pre-selected parameters (either through automation or through thorough experimentation)
  • Model deployment: This is the first step that I will dive a bit more deeply into below.

Model deployment

  • Generate a configuration file for model details
  • Save the model
  • Freeze the model
  • Serve the model

Generate configuration file

  • Make it easier to tell the mobile device what features to extract and use for a newly downloaded model.
  • Contain information to keep track of what model is deployed on which device
  • Contain information on what the numeric output of the model means — such as corresponding labels
  • Contain information on how to access the relevant parts of the model — such as the names of the input and output layers of a neural network

Save the model

  • A meta file: Contains the graph information, in other words, it describes the architecture of the model
  • An index file — Metadata of each tensor that is part of the graph
  • A data file — Contains the values of the variables, such as the values of the weights.

Freeze the model

Serve the model

  1. Get the latest configuration for the OS — the service provides this in JSON format
  2. If the latest configuration has a higher version than the local backup configuration
  3. Then, download the latest model — the service provides this in a zip file
  4. And replace the local model and configuration with the latest downloaded
  5. If the latest configuration does not have a higher version than the local one OR there is no internet connection
  6. Use the locally stored model
  • What features to extract
  • The name of the input layer
  • The name of the output layer
  • The meaning of the output

Additional considerations

  • Model files are corrupted through network transfer
  • Configurations don’t fit the model deployed (for example 3 outputs, but only 2 described)
  • Features described are not understood by the mobile device

Conclusion

--

--

--

Data scientist, software engineer, poet, writer, blogger, ammature painter

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Would this clothing fit me?

Support Vector Machine - the big ideas

My first ever article.(Machine Learning — Supervised Learning)

One Class Learning in Manufacturing: Autoencoder and Golden Units Baselining

MACHINE LEARNING:A MEANS TO PREDICT POTENTIAL CUSTOMER CHURN

Overall Equipment Effectiveness and Topic Modeling

Reproducible model training: deep dive

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Kristina Georgieva

Kristina Georgieva

Data scientist, software engineer, poet, writer, blogger, ammature painter

More from Medium

Elasticsearch Analyzers

How to prevent sensitive data leakages through code repositories

How does Java run the file?

What is Recommendation/Suggestion System?